Control method for self-moving device, and electronic device

Abstract

The present application relates to a control method for a self-moving device, and an electronic device. A self-moving device comprises a non-scanning single-frame pulsed imaging sensor, which sensor is configured to collect surrounding environment information of the self-moving device, so as to obtain three-dimensional point cloud data. The control method for a self-moving device comprises: acquiring three-dimensional point cloud data; and on the basis of the three-dimensional point cloud data, determining target information, wherein the target information comprises at least one of a target attribute, an environmental terrain, weather conditions, and an operating environment state of a sensor. Another control method for a self-moving device comprises: acquiring three-dimensional point cloud data, and motion state data detected by a state detection unit; according to time stamps, performing alignment on the three-dimensional point cloud data and the motion state data, so as to obtain three-dimensional point cloud data and motion state data at the same moment; then, projecting the three-dimensional point cloud data and motion state data at the same moment onto the same coordinate system, so as to obtain multi-source data at the same moment and in the same coordinate system; and on the basis of the multi-source data, performing computation to obtain pose information of the self-moving device, and / or constructing an environmental point-cloud map.

Description

Control methods and electronic devices for self-moving devices Technical Field

[0001] This application relates to the field of self-moving device control technology, and in particular to a control method and electronic device for self-moving devices. Background Technology

[0002] Self-moving devices, such as automatic lawnmowers, can complete tasks autonomously without direct human control or operation, significantly reducing reliance on manpower and thus gaining increasing popularity among users. With the rapid development of autonomous self-moving device technology, these devices are evolving from simple random path planning to environmentally aware intelligent devices.

[0003] In related technologies, environmental perception methods for self-moving devices mainly fall into two categories: one uses ultrasonic sensors, infrared sensors, or single-line lidar; the other uses multi-sensor fusion technology, combining data from multiple different types of sensors. However, none of the above perception methods achieve three-dimensional environmental perception, which limits their ability to handle complex operational scenarios. Summary of the Invention

[0004] This application provides a control method and electronic device for a self-moving device, which can realize three-dimensional environmental perception.

[0005] In a first aspect, this application provides a control method for a self-moving device, the self-moving device including a non-scanning single-frame pulse imaging sensor, the non-scanning single-frame pulse imaging sensor being configured to collect information about the surrounding environment of the self-moving device and obtain three-dimensional point cloud data.

[0006] The control method for the self-moving device includes:

[0007] Acquire the three-dimensional point cloud data;

[0008] Based on the three-dimensional point cloud data, determine the target information;

[0009] The target information includes at least one of the following: target attributes, environmental terrain, weather conditions, and sensor operating environment conditions.

[0010] In some embodiments, the control method for the self-moving device further includes:

[0011] Based on the target information, the operating status of the self-moving device is controlled;

[0012] The operating state of the self-moving device includes the motion state of the self-moving device and / or the working state of the self-moving device.

[0013] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0014] Based on the three-dimensional point cloud data, determine the target feature parameters;

[0015] The target information is determined based on the target feature parameters.

[0016] In some implementations, determining the target information based on the target feature parameters includes:

[0017] The target information is determined based on whether the target feature parameters meet the preset target conditions.

[0018] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0019] Based on preset feature conditions, feature points are extracted from the three-dimensional point cloud data to obtain candidate point set data;

[0020] The target information is determined based on the candidate point set data;

[0021] The target information includes at least one of the following: target attributes and environmental terrain.

[0022] In some embodiments, the control method for the self-moving device further includes:

[0023] In response to the satisfaction of preset detection conditions, the target information is determined based on the three-dimensional point cloud data.

[0024] In some embodiments, the control method for the self-moving device further includes:

[0025] According to preset filtering conditions, the three-dimensional point cloud data is preprocessed to filter out invalid points and obtain simplified point cloud data.

[0026] The target information is determined based on the simplified point cloud data.

[0027] In some implementations, the three-dimensional point cloud data includes the spatial coordinates of the points, reflection intensity, and row and column indices.

[0028] In some implementations, determining the target information based on the 3D point cloud data is configured as follows:

[0029] The target information is determined solely based on the three-dimensional point cloud data.

[0030] In some implementations, the target information includes at least two of the following: target attributes, environmental terrain, weather conditions, and sensor operating environment conditions.

[0031] In some embodiments, the non-scanning single-frame pulse imaging sensor is positioned toward the direction of travel of the self-moving device to acquire environmental information in front of the self-moving device.

[0032] In some embodiments, the horizontal field of view of the non-scanning single-frame pulse imaging sensor is 75° to 150°, and the vertical field of view is 15° to 90°.

[0033] In some embodiments, the non-scanning single-frame pulse imaging sensor includes a laser emitter array and a laser receiver array that cooperates with the laser emitter array to acquire the three-dimensional point cloud data;

[0034] The self-moving device includes a body, and the laser emitter array and the laser receiver array are respectively fixedly disposed relative to the body.

[0035] In some embodiments, the non-scanning single-frame pulse imaging sensor includes a solid-state area array lidar.

[0036] In some embodiments, the self-moving device is an automatic lawnmower.

[0037] In some implementations, the target attribute includes at least one of obstacles, passable areas, and charging piles;

[0038] The obstacles include at least one of static obstacles and dynamic obstacles.

[0039] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0040] Based on the three-dimensional point cloud data, the obstacles and the passable areas are identified according to the first preset conditions;

[0041] The first preset condition includes at least one of the following: geometric feature condition, penetration condition, reflection intensity condition, ground continuity condition, and morphological stability condition in multi-frame data.

[0042] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0043] Based on the three-dimensional point cloud data, the dynamic obstacle is identified according to the second preset condition;

[0044] The second preset condition includes at least one of the following: point cloud contour change condition, position change condition, size change condition, distribution density change condition, local curvature change condition, motion speed condition, motion acceleration condition, motion direction consistency condition, trajectory continuity condition, and reflection intensity change condition in consecutive frames.

[0045] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0046] Based on the three-dimensional point cloud data, the charging pile is identified according to the third preset condition;

[0047] The third preset condition includes at least one of the following: geometric feature condition of the target point cloud, reflection intensity condition, height condition, and distance condition.

[0048] In some embodiments, the environmental terrain includes at least one of cliffs and slopes.

[0049] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0050] Based on the three-dimensional point cloud data, the cliff is identified according to the fourth preset condition;

[0051] The fourth preset condition includes at least one of the following: point height condition, point distance condition, vertical height change condition of a point in a local area, and horizontal distance change condition of a point in a local area.

[0052] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0053] Based on the three-dimensional point cloud data, the ramp is identified according to the fifth preset condition;

[0054] The fifth preset condition includes at least one of the following: point height condition, Euclidean distance between adjacent points condition, point cloud direction change condition, and tilt angle condition of point cloud extension direction.

[0055] In some embodiments, the weather condition includes at least one of rainy weather and foggy weather.

[0056] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0057] Based on the three-dimensional point cloud data, the foggy weather is identified according to the sixth preset condition;

[0058] The sixth preset condition includes at least one of the following: the number of fog points in the three-dimensional point cloud data, the number of foggy zones, the proportion of foggy zones, the number of distant points, the number of ground points, and the number of invalid points.

[0059] The fog point is determined based on the distance between the point and the sensor origin and the reflection intensity of the point;

[0060] The horizontal field of view of the non-scanning single-frame pulse imaging sensor is divided into multiple zones, and each zone is determined to be a foggy zone based on the number of fog spots in each zone.

[0061] The percentage of foggy zones is the ratio of the number of foggy zones to the total number of zones.

[0062] The distant point is defined as a point whose distance from the origin of the sensor is greater than or equal to the distant distance threshold.

[0063] The ground point is defined as a point whose height is less than the ground height threshold.

[0064] The invalid point is a point where all spatial coordinate components are 0 and the reflection intensity is also 0.

[0065] In some embodiments, the fog point is: an effective point whose distance from the sensor origin is less than or equal to a near-distance threshold and whose reflection intensity is less than or equal to a low-intensity threshold;

[0066] The effective point is defined as a point in which at least one of the coordinate components of the spatial coordinates is not zero, or the reflection intensity is not zero.

[0067] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0068] Based on the three-dimensional point cloud data, the rainy day is identified according to the seventh preset condition;

[0069] The seventh preset condition includes at least one of the following: the number of near-distance low-intensity points in the three-dimensional point cloud data, the number of invalid points, the number of far-distance points, the proportion of near-distance low-intensity point cloud, and the point cloud distribution variance.

[0070] The near-distance low-intensity point is defined as a point whose distance from the sensor origin is less than the near-distance threshold and whose intensity is less than the low-intensity threshold.

[0071] The invalid point is a point where all spatial coordinate components are 0 and the reflection intensity is 0.

[0072] The distant point is defined as a point whose distance from the origin of the sensor is greater than a distant distance threshold.

[0073] The proportion of near-range low-intensity point clouds is the ratio of the number of valid points whose distance from the sensor origin is less than the near-range threshold and whose intensity is less than the low-intensity threshold, to the total number of valid points whose distance from the sensor origin is less than the near-range threshold.

[0074] The point cloud distribution variance is: the horizontal field of view of the non-scanning single-frame pulse imaging sensor is divided into multiple partitions, and the point cloud distribution variance is used to characterize the uniformity of the distribution of effective points in each partition.

[0075] The valid point is a point whose coordinate components in the spatial coordinates are at least non-zero, or whose reflection intensity is non-zero.

[0076] In some embodiments, the sensor operating environment state includes the sensor being obstructed.

[0077] In some implementations, determining the target information based on the three-dimensional point cloud data includes:

[0078] Based on the three-dimensional point cloud data, the non-scanning single-frame pulse imaging sensor is identified as being occluded according to the eighth preset condition.

[0079] The eighth preset condition includes at least one of the following: invalid point quantity condition, invalid point percentage condition, occluded point quantity condition, occluded point percentage condition, and non-occluded point quantity condition.

[0080] The invalid point is a point where all spatial coordinate components are 0 and the reflection intensity is also 0.

[0081] The invalid point ratio is the ratio of the number of invalid points to the total number of points in a single frame of the 3D point cloud data.

[0082] The occluded point and the unoccluded point are determined based on the distance between the point and the sensor origin and / or the spatial coordinates of the point. The distance between the occluded point and the sensor origin is less than the distance between the unoccluded point and the sensor origin.

[0083] The occlusion point ratio is the ratio of the number of occlusion points to the number of valid points in a single frame of the 3D point cloud data.

[0084] The valid point is a point where at least one of the coordinate components of the spatial coordinates is not 0, or the reflection intensity is not 0.

[0085] In some embodiments, the occlusion point includes: a point whose distance from the sensor origin satisfies a close proximity condition;

[0086] And / or, the unobstructed point is: a point whose distance from the origin of the sensor is greater than or equal to the far distance threshold.

[0087] Secondly, this application also provides a control method for a self-moving device, the self-moving device comprising:

[0088] A non-scanning single-frame pulse imaging sensor is configured to acquire environmental information about the self-moving device to obtain three-dimensional point cloud data; and

[0089] The state detection unit is configured to detect the motion state data of the self-moving device;

[0090] The control method for the self-moving device includes:

[0091] Acquire the 3D point cloud data and the motion state data;

[0092] Align the 3D point cloud data and the motion state data according to the timestamp to obtain the 3D point cloud data and the motion state data at the same moment;

[0093] The three-dimensional point cloud data and the motion state data at the same moment are projected onto the same coordinate system to obtain multi-source data at the same moment and in the same coordinate system;

[0094] Based on the multi-source data, the pose information of the self-moving device is calculated and / or an environmental point cloud map is constructed.

[0095] In some embodiments, the state detection unit includes an inertial measurement unit;

[0096] The motion state data includes acceleration and angular velocity.

[0097] In some embodiments, the state detection unit further includes a wheel odometer;

[0098] The motion state data also includes displacement and orientation angle.

[0099] In some implementations, calculating the pose information of the self-moving device based on the multi-source data, and / or constructing an environmental point cloud map, includes:

[0100] Based on the multi-source data, a state estimation is performed using a fusion algorithm to output the pose information of the self-moving device and / or an environmental point cloud map.

[0101] In some embodiments, the control method for the self-moving device further includes:

[0102] Get global map information;

[0103] Global positioning information is obtained based on the 3D point cloud data, the motion state data, and the global map information.

[0104] Thirdly, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the above-described control method for the self-moving device. Attached Figure Description

[0105] Figure 1 is a schematic diagram of the working scenario of a self-moving device according to an embodiment of this application;

[0106] Figure 2 is a schematic diagram of the structure of a self-moving device according to an embodiment of this application;

[0107] Figure 3 is a flowchart of a control method for a self-moving device according to an embodiment of this application;

[0108] Figure 4 is a flowchart of a method for determining target attribute information based on three-dimensional point cloud data according to an embodiment of this application;

[0109] Figure 5 is a flowchart of a method for determining cliff terrain based on three-dimensional point cloud data according to an embodiment of this application;

[0110] Figure 6 is a flowchart of a method for determining cliff terrain based on three-dimensional point cloud data according to another embodiment of this application;

[0111] Figure 7 is a flowchart of a method for identifying target objects based on three-dimensional point cloud data according to an embodiment of this application;

[0112] Figure 8 is a schematic diagram of three-dimensional point cloud data according to an embodiment of this application;

[0113] Figure 9 is a schematic diagram of three-dimensional point cloud data according to another embodiment of this application;

[0114] Figure 10 is a flowchart of a method for identifying target objects based on three-dimensional point cloud data according to a specific embodiment of this application;

[0115] Figure 11 is a flowchart of a method for identifying foggy weather based on three-dimensional point cloud data according to an embodiment of this application;

[0116] Figure 12 is a flowchart of a method for identifying foggy weather based on three-dimensional point cloud data according to another embodiment of this application;

[0117] Figure 13 is a flowchart of a method for identifying foggy weather based on three-dimensional point cloud data according to another embodiment of this application;

[0118] Figure 14 is a flowchart of a method for identifying occlusion based on three-dimensional point cloud data according to an embodiment of this application;

[0119] Figure 15 is a flowchart of a method for identifying occlusion based on three-dimensional point cloud data according to another embodiment of this application;

[0120] Figure 16 is a flowchart of a method for identifying occlusion based on three-dimensional point cloud data according to another embodiment of this application;

[0121] Figure 17 is a flowchart of a control method for a self-moving device according to another embodiment of this application;

[0122] Figure 18 is a flowchart of a control method for a self-moving device according to another embodiment of this application;

[0123] Figure 19 is a flowchart of a control method for a self-moving device according to another embodiment of this application;

[0124] Figure 20 is a flowchart of an iterative correction process according to an embodiment of this application;

[0125] Figure 21 is a flowchart of a relocation method according to an embodiment of this application;

[0126] Figure 22 is a schematic electrical block diagram of a self-moving device according to an embodiment of this application;

[0127] Figure 23 is a schematic electrical block diagram of a self-moving device according to another embodiment of this application;

[0128] Figure 24 is a schematic circuit block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0129] The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the application. For example, terms such as "upper," "lower," "front," and "rear" that indicate orientation or positional relationship are based solely on the orientation or positional relationship shown in the accompanying drawings and are used only for the convenience of describing the application and simplifying the description, and are not intended to indicate or imply that the device / component referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the application.

[0130] Self-moving devices use sensors to perceive information about their surroundings, such as obstacles, passable areas, terrain, and weather conditions, in order to intelligently control the operation of the self-moving device based on the identified information.

[0131] Referring to Figure 1, the self-moving device 10, such as an automatic lawnmower, operates within its designated work area. When the self-moving device 10 is not in operation or needs charging, it returns to the charging station 20 to dock or recharge. The self-moving device 10 is equipped with sensors that perceive information about its surrounding environment, identifying obstacles, passable areas, terrain, weather conditions, etc., to intelligently control its operation based on this information. Of course, the self-moving device can also be an indoor smart home device, such as an automatic robot vacuum cleaner or a robotic vacuum cleaner.

[0132] In related technologies, environmental perception methods for self-moving devices mainly fall into two categories: one uses ultrasonic sensors, infrared sensors, or single-line LiDAR; the other uses multi-sensor fusion technology, combining data from multiple different types of sensors. None of these perception methods achieve three-dimensional environmental perception, which limits their effectiveness in handling complex work scenarios. Mainstream single-line LiDAR can only acquire two-dimensional planar information, failing to effectively identify low-lying obstacles, and the lack of three-dimensional environmental perception easily leads to collisions. Infrared sensor-based ranging solutions suffer from significant ranging errors in lighting conditions, causing frequent false braking in areas with overlapping shadows, severely impacting work efficiency and exhibiting poor adaptability to complex terrain. Traditional solutions cannot construct three-dimensional elevation maps of the work area, making equipment prone to tipping over on sloping terrain and unable to achieve accurate grass height detection and adaptive cutting. Furthermore, multi-sensor fusion solutions suffer from high computational resource consumption, high light sensitivity, and a sharp increase in hardware costs.

[0133] To address the aforementioned problems, one embodiment of this application provides a control method for a self-moving device. Referring to Figure 2, the self-moving device 10 includes a non-scanning single-frame pulse imaging sensor 101 (also known as a detection module for multi-dimensional imaging without mechanical scanning). The non-scanning single-frame pulse imaging sensor 101 is configured to acquire information about the surrounding environment of the self-moving device 10, obtaining three-dimensional point cloud data. Referring to Figure 3, the control method for the self-moving device includes:

[0134] S101. Obtain 3D point cloud data;

[0135] S102. Determine target information based on 3D point cloud data; wherein, target information includes at least one of target attributes, environmental terrain, weather conditions, and sensor operating environment conditions.

[0136] Non-scanning single-frame pulse imaging sensors refer to sensors that can achieve multi-dimensional imaging with a single laser pulse emission without mechanical scanning. They can acquire three-dimensional information within the entire field of view using two-dimensional or multi-dimensional arrays of laser emitters and receivers, achieving instantaneous imaging. In specific embodiments, this can be a solid-state area array lidar (also known as Flash LiDAR). Compared to traditional mechanical lidar that uses rotating mirrors to scan the surrounding environment, solid-state area array lidar can achieve faster and more accurate scanning by using solid-state emitter and receiver arrays.

[0137] It is understandable that the 3D point cloud data acquired by the non-scanning single-frame pulse imaging sensor has high accuracy, which can improve the 3D perception performance. Compared with mechanical scanning sensors, the data real-time performance is higher. Based on the 3D point cloud data acquired by the non-scanning single-frame pulse imaging sensor, target information can be detected more accurately and in real time.

[0138] Furthermore, the control method for the self-moving device also includes: controlling the operating state of the self-moving device based on target information; wherein the operating state of the self-moving device includes the movement state and / or the working state of the self-moving device. For example, taking an automatic lawnmower as an example, based on the detection of an obstacle, the automatic lawnmower is controlled to perform obstacle avoidance; based on the detection of a passable area, the lawnmower is controlled to pass normally; based on the detection of cliff terrain, the automatic lawnmower is controlled to avoid cliff areas; based on the detection of weather conditions such as fog or rain, the automatic lawnmower is controlled to stop moving or return to the charging station; based on the detection that the sensor surface is obstructed, the automatic lawnmower is controlled to stop moving.

[0139] In some embodiments, the control method for the self-moving device further includes: determining target information based on 3D point cloud data in response to meeting preset detection conditions. It is understood that by using preset detection conditions, the confirmation of specific target information can be initiated only in specific scenarios, saving computing power and improving the real-time performance of target information detection and control. For example, cliff detection is initiated only when the self-moving device approaches the working boundary; since the area inside the working boundary is a defined working area and there are no cliffs within the working area, initiating cliff detection only when the self-moving device approaches the working boundary can save computing power for the detection of other target information.

[0140] In some embodiments, step S102 is configured to determine target information based solely on 3D point cloud data. It can be understood that this embodiment achieves target information recognition based solely on 3D point cloud data acquired by a non-scanning single-frame pulse imaging sensor. Compared to multi-sensor fusion schemes, this significantly reduces hardware costs and power consumption, effectively lowering equipment production and usage costs.

[0141] In some embodiments, target information includes at least two of the following: target attributes, environmental terrain, weather conditions, and sensor operating environment conditions. It can be understood that this embodiment, based on 3D point cloud data acquired by a non-scanning single-frame pulse imaging sensor, enables the identification of multiple targets, environmental terrain, weather conditions, and sensor operating environment conditions. This allows for sensor reuse and extended functionality, effectively reducing equipment costs, ensuring that self-moving devices can adapt to various complex environments, and improving environmental adaptability.

[0142] A non-scanning single-frame pulse imaging sensor is positioned facing the traveling direction of the self-moving device to acquire environmental information in front of the device. Specifically, it can be mounted in front of or on top of the self-moving device. The horizontal field of view of the non-scanning single-frame pulse imaging sensor is 75°–150°, and the vertical field of view is 15°–90°. It is understood that these field of view angles are determined by the characteristics of the sensor. The non-scanning single-frame pulse imaging sensor is configured with preferred horizontal and vertical field of view angles to balance detection range and high detection accuracy. Compared to traditional single-line radar solutions, the detection dimension is expanded from a two-dimensional plane to three-dimensional space, resulting in higher accuracy in target information inspection.

[0143] The non-scanning single-frame pulse imaging sensor 101 includes a solid-state area array lidar. Referring to Figure 2, the non-scanning single-frame pulse imaging sensor 101 includes a laser emitter array and a laser receiver array that cooperates with the laser emitter array to acquire three-dimensional point cloud data; the self-moving device includes a body 102, with the laser emitter array and laser receiver array fixedly disposed relative to the body 102. It can be understood that, compared to scanning sensors, the non-scanning single-frame pulse imaging sensor, being fixed to the body, has higher mechanical strength and a longer lifespan.

[0144] The 3D point cloud data includes the spatial coordinates (x, y, z) of each point, the reflection intensity, and the row and column indices (row, col). The spatial coordinates (x, y, z) represent the geometric position of each point in 3D space; the reflection intensity represents the echo signal intensity of the target surface corresponding to each point; the row and column indices (row, col) are calculated based on the vertical pitch angle of each laser point and the horizontal azimuth angle of each laser point, thus constructing a mapping relationship from 2D indices to 3D coordinates.

[0145] In some embodiments, step S102 includes: determining target feature parameters based on 3D point cloud data; and determining target information based on the target feature parameters. Specifically, target information is determined based on whether the target feature parameters meet preset target conditions.

[0146] For example, based on 3D point cloud data, target feature parameters for fog detection are determined, including the number of fog points, the number of fog zones, the number of distant points, the number of ground points, and the number of invalid points. Fog points are determined based on the distance between the point and the sensor origin and the point's reflection intensity. The sensor's horizontal field of view is divided into multiple zones, and each zone is determined to be a fog zone based on the number of fog points within it. Distant points are defined as points whose distance from the sensor origin is greater than or equal to a distant distance threshold. Ground points are defined as points whose height is less than a ground height threshold. Invalid points are points whose spatial coordinate components are all 0 and whose reflection intensity is also 0. The sensor origin is the origin of the sensor coordinate system established based on a non-scanning single-frame pulse imaging sensor. Specifically, based on the spatial coordinates (x, y, z) of a point, the distance between the point and the sensor origin, and the point's height, can be calculated. Fog points can be identified and their counts can be determined based on the distance between the point and the sensor origin and the point's reflection intensity. The number of fog points within each zone can be used to determine whether each zone is a foggy zone and the number of foggy zones can be determined. Distant points can be identified and their counts can be determined based on the distance between the point and the sensor origin. Ground points can be identified and their counts can be determined based on the point's height. Invalid points can be identified and their counts can be determined based on the point's spatial coordinates and reflection intensity. Whether the number of fog points, foggy zones, distant points, ground points, and invalid points meets preset target conditions determines whether the current weather condition is foggy. Specifically, the preset target conditions include: the number of fog points is greater than or equal to a fog point count threshold; the number of foggy zones is greater than or equal to a zone count threshold; the number of distant points is less than or equal to a distant point count threshold; the number of ground points is less than or equal to a ground point count threshold; and the number of invalid points is greater than or equal to an invalid point count threshold. If the above conditions are met, the current weather condition is determined to be foggy.

[0147] In some embodiments, step S102 includes: extracting feature points from the 3D point cloud data according to preset feature conditions to obtain candidate point set data; and determining target information based on the candidate point set data; wherein the target information includes at least one of target attributes and environmental terrain. It can be understood that by extracting candidate point set data that conforms to the target information features and filtering out non-candidate points, the computational load can be reduced and real-time performance improved.

[0148] As an example, when identifying passable areas for automatic lawnmowers, points suspected to be passable areas are first extracted based on 3D point cloud data as a candidate set of passable areas. Then, based on the candidate set of passable areas, grass areas, low obstacles, passable ramps, and impassable steep slopes are further identified, and finally, grass areas and passable ramps are determined to be passable areas.

[0149] As another example, when performing cliff terrain detection, the three-dimensional point cloud data is mapped to a two-dimensional grid. If the coordinate component z of the current grid point satisfies that the spatial coordinate component z>0 (higher than the height of the non-scanning single-frame pulse imaging sensor, which is likely a protruding obstacle) or the distance from the sensor origin is greater than 1m (a distant point, which has no impact on near obstacle avoidance), then the point is determined to be a non-cliff candidate point. Subsequent further detection is skipped, and only cliff candidate points are extracted to obtain cliff candidate point set data. Based on the cliff candidate point set data, cliff information is detected.

[0150] In some embodiments, the control method for the self-moving device further includes: preprocessing the three-dimensional point cloud data according to preset filtering conditions to filter out invalid points and obtain simplified point cloud data; and determining target information based on the simplified point cloud data. It can be understood that by filtering out invalid points and retaining only valid points for detection, detection accuracy and robustness can be improved. For example, when performing cliff terrain detection, a single frame of original three-dimensional point cloud data is preprocessed, and invalid points in the area behind the sensor (with a horizontal distance of less than 0.1m from the sensor origin) and in the sensor's rear region are filtered out according to two preset filtering conditions, retaining only valid points for detection.

[0151] The target attributes include at least one of obstacles, passable areas, and charging piles; wherein obstacles include at least one of static obstacles and dynamic obstacles.

[0152] In some embodiments, step S102 includes: identifying obstacles and passable areas based on three-dimensional point cloud data and according to a first preset condition; wherein the first preset condition includes at least one of geometric feature condition, penetration condition, reflection intensity condition, ground continuity condition, and morphological stability condition in multi-frame data.

[0153] For example, an obstacle and passable region identification method based on 3D point cloud data, integrating geometric features, penetration, reflection intensity, ground continuity, and multi-frame morphological stability, includes: 1) Geometric feature conditions: Since obstacles have significant height abrupt changes or non-planar structures in local areas, the height difference between each point and the fitted ground is calculated by fitting the point cloud to the ground. If the height is high and the angle between the local normal vector of the point cloud and the vertical direction exceeds an angle threshold (e.g., 30°), it is marked as a potential obstacle; conversely, areas with low height and near-vertical normal vectors are marked as passable candidate regions; 2) Penetration conditions: Since passable regions allow laser penetration (e.g., sparse grass), while obstacles... (e.g., solid wall) blocking pulses manifests as a sudden change in point cloud density. By projecting the point cloud onto a horizontal grid (e.g., 0.1m × 0.1m), the number of points and minimum height within each grid are counted. If the number of points in the grid is below the penetration threshold (e.g., 3 points) and the minimum height is close to the ground, it is determined to be a passable area (e.g., sparse grass). If the number of points is dense and the minimum height is significantly higher than the ground, it is determined to be an obstacle. 3) Reflection intensity conditions: Since different materials have different laser reflection intensities, low-intensity areas may be absorbing materials or distant targets, while high-intensity areas may be highly reflective surfaces, requiring distance correction. By normalizing the reflection intensity by distance (compensating for attenuation), if the normalized intensity is low and the number of points is low, it is determined to be a passable area (e.g., sparse grass). Low cloud heights may indicate passable surfaces (such as asphalt); high intensity and discontinuous spatial morphology indicate reflective interference obstacles (such as mirrors or puddles), requiring secondary verification based on geometric conditions; 4) Ground continuity conditions: Passable areas are spatially continuous and smooth ground surfaces; obstacles disrupt this continuity, creating abrupt depth edges; using graph cut or region growing algorithms, expand from the lowest point, judging the distance and height difference between adjacent points: if the horizontal distance is below a threshold (e.g., 0.5m) and the height difference is below a threshold (e.g., 0.1m), they are considered to belong to the same continuous ground; if a jump occurs with a height difference greater than the threshold, the point on the higher side of the jump is marked as an obstacle. The boundary and lower side remain passable areas; 5) Morphological stability conditions in multi-frame data: Static obstacles (such as walls and vehicles) are stable in shape and position in multi-frame point clouds; dynamic obstacles (such as pedestrians and moving vehicles) are unstable; passable ground is usually statically stable; based on multi-frame point cloud data, the occupancy probability and height variance are calculated for the same spatial grid (or voxel). The occupancy probability is the proportion of frames occupied. If the occupancy probability > 0.8 and the height variance is small, it is judged as a static obstacle; if the occupancy probability is between 0.3 and 0.7 or the height variance is large, it is judged as a dynamic obstacle, which does not affect the passable area decision; if the occupancy probability < 0.3 and the average height is low, it is a passable area. It can be understood that the above conditions can all achieve the identification of obstacles and passable areas to a certain extent, but the identification accuracy of a single condition judgment is lower than that of multi-condition fusion, and the reliability is lower.Therefore, in some embodiments, by optionally fusing the above conditions, the recognition accuracy and reliability can be effectively improved.

[0154] In one specific embodiment, as shown in Figure 4, obstacle information within the visible range is roughly extracted based on 3D point cloud data to obtain simplified feature data. The roughly extracted obstacle information refers to extracting obstacle information within a preset distance range and a preset height range to obtain simplified obstacle feature data. The preset distance can be 0.2m-1.5m, 0.2m-1.2m, or 0.2-1m; the preset height range can be 0-0.4m, 0-0.6m, or 0-0.8m. In an alternative embodiment, the preset height can be 1.1-1.5 times the height H of the self-moving device, i.e., the preset height range is 0-1.1H, 0-1.2H, 0-1.3H, 0-1.4H, or 0-1.5H. The self-moving device extracts obstacle features and outputs obstacle information based on filtering and clustering algorithms. Furthermore, to improve the accuracy of the output obstacle information, multiple consecutive frames of point cloud data can be processed. If obstacle information is present in all of the multiple consecutive frames of point cloud data, then the obstacle information is output. The number of frames in the multiple consecutive frames of point cloud data can be 2, 3, 4, 5, etc., and can be set according to actual needs. Obstacle recognition based on 3D point cloud data acquired by a non-scanning single-frame pulse imaging sensor can detect target information more accurately and in real-time, especially significantly improving the accuracy of low-lying obstacle recognition.

[0155] In some embodiments, step S102 includes: identifying dynamic obstacles based on three-dimensional point cloud data according to a second preset condition; wherein the second preset condition includes at least one of the following: point cloud contour change condition, position change condition, size change condition, distribution density change condition, local curvature change condition, motion speed condition, motion acceleration condition, motion direction consistency condition, trajectory continuity condition, and reflection intensity change condition.

[0156] For example, a method for identifying dynamic obstacles based on 3D point cloud data, which integrates conditions for point cloud contour change, position change, size change, distribution density change, local curvature change, motion velocity, motion acceleration, motion direction consistency, trajectory continuity, and reflection intensity change in consecutive frames, includes: 1) Contour change condition: Because dynamic obstacles undergo significant changes in their contour due to changes in their posture or observation perspective during movement, while the contour of static objects remains stable; by calculating the convex hull overlap rate of the same target point cloud in adjacent frames, if the overlap rate is lower than a threshold (e.g., 0.6), it is determined to be a dynamic obstacle; 2) Position change condition: Because the centroid position of a dynamic target in the global coordinate system changes significantly over time; by calculating the centroid displacement, if the centroid displacement exceeds the position change threshold calculated by velocity (e.g., 0.2 m / frame), it is determined to be a dynamic obstacle; 3) Size change condition: Because dynamic... When a target moves closer to or further away from the sensor, the point cloud size changes. By combining distance correction, the estimated true physical size is calculated. If the estimated length / width change rate between adjacent frames is greater than 30%, it is determined to be a dynamic obstacle. 4) Distribution density change condition: As a dynamic target moves, the density distribution of the point cloud in the local space changes (e.g., it becomes denser when it gets closer). The distribution density is characterized by the average point spacing or voxel occupancy rate of the target point cloud. The density change coefficient is calculated. If the density change coefficient is greater than 0.5, it is determined to be a dynamic obstacle. 5) Local curvature change condition: Due to the change in the incident angle of the target surface relative to the sensor caused by motion, the local curvature estimation fluctuates. By calculating the local curvature of each point (e.g., PCA eigenvalue ratio), if the inter-frame change in the average curvature of the point cloud within the target is greater than a preset threshold, it is determined to be a dynamic obstacle. 6) Motion speed condition: The motion speed is obtained by dividing the position difference by the time interval. If the motion speed meets the preset condition (e.g., greater than 0.5), it is determined to be a dynamic obstacle.7) Motion acceleration condition: Since continuous non-zero or sudden acceleration indicates the active movement of the target, the acceleration of the target point cloud is calculated. If the acceleration meets the preset condition (e.g., greater than 2 m / s²), it is determined to be a dynamic obstacle; 8) Motion direction consistency condition: Since the motion direction of dynamic targets is relatively consistent in a short period of time, while the direction change of static or random noise is chaotic; the displacement direction angle of continuous frames is calculated. If the standard deviation of the direction angle change is less than 30°, it meets the consistency condition and is determined to be a dynamic obstacle; 9) Trajectory continuity condition: Since a dynamic target should have a smooth and predictable trajectory across multiple frames, the next frame's position is predicted using Kalman filtering or polynomial fitting. If the matching error with the actual position is less than a threshold, the trajectory is considered continuous, and the target is identified as a dynamic obstacle. 10) Reflection intensity change condition: Due to target motion causing changes in the laser incident angle or directional reflection from the surface material, intensity fluctuations occur. By statistically analyzing the inter-frame variance of the average normalized intensity value of the point cloud within the target, if the variance is greater than the intensity change threshold and the velocity condition is also met, the target is identified as a dynamic obstacle. It is understandable that the above conditions can all achieve dynamic obstacle recognition to some extent, but judging based on a single condition results in lower recognition accuracy and reliability compared to fusion of multiple conditions. Therefore, in some embodiments, selectively fusing the above conditions can effectively improve recognition accuracy and reliability.

[0157] In some embodiments, step S102 includes: identifying charging piles based on three-dimensional point cloud data and according to a third preset condition; wherein the third preset condition includes at least one of the following: geometric feature condition of the target point cloud, reflection intensity condition, height condition, and distance condition.

[0158] The self-moving device needs to be able to sense the charging pile in order to accurately dock with it. Two regularly shaped reflectors are placed on the charging pile; by observing the difference in reflection intensity at points, the approximate location of the charging pile can be roughly identified. As shown in Figure 5, a candidate set of charging pile points is roughly extracted based on the reflection intensity information of points in the 3D point cloud data. Based on the geometric features of the two reflectors, such as the distance between their center points and the height of the points, the accurate location information of the charging pile is further determined. Then, the surrounding contour points of the charging pile are extracted to obtain its structural shape, i.e., the target contour data. A clustering algorithm is then used to accurately identify the charging pile, outputting its pose information, thereby finding the accurate docking point.

[0159] The environmental terrain includes at least one of cliffs and slopes.

[0160] When self-moving equipment travels within a work area (lawn, yard, room), in addition to identifying conventional obstacles, it also needs to identify dangerous areas such as steps, pits, ditches, and cliffs to prevent falls, rollovers, and other safety accidents. In the field of cliff detection for self-moving equipment, mainstream technologies largely rely on infrared detection. The core principle is to use infrared sensors to emit infrared signals, receive reflected signals, and determine the presence of a cliff based on the intensity of the reflected light. However, infrared sensing can only detect the presence of a cliff, not its specific parameters. Besides infrared detection, multi-sensor fusion (such as combining depth images, brightness information, and infrared sensing) cliff detection methods also exist. While these methods attempt to improve upon the shortcomings of infrared detection by obtaining specific cliff parameters, they require integrating data from multiple sensors. This increases hardware costs and structural complexity, and necessitates complex fusion calculations of multi-source data, resulting in slow detection speeds. This makes it difficult to meet the operational needs of self-moving equipment and adapt to dynamic obstacle avoidance scenarios in complex terrain.

[0161] In some embodiments, step S102 includes: identifying cliffs based on three-dimensional point cloud data according to a fourth preset condition; wherein the fourth preset condition includes at least one of: point height condition, point distance condition, vertical height change condition of points in a local area, and horizontal distance change condition of points in a local area.

[0162] Referring to Figure 5, the control method for the self-moving device provided in this embodiment is used for cliff detection. Taking the non-scanning single-frame pulse imaging sensor as a solid-state area array lidar as an example, it specifically includes the following steps 411-413.

[0163] Step 411: Obtain 3D point cloud data.

[0164] In some embodiments, the 3D point cloud data is the raw 3D point cloud data collected by the lidar, without any processing.

[0165] In other embodiments, the 3D point cloud data is the effective 3D point cloud data obtained after preprocessing the raw 3D point cloud data acquired by the LiDAR. Specific preprocessing methods include:

[0166] (1) Filter out laser points whose horizontal distance from the lidar is less than the preset value, eliminate reflection points generated by the lidar's own shell, self-moving device shell and other structures, as well as unreliable measurement values ​​in the nearby blind zone.

[0167] (2) Filter out the laser points and noise points located behind the lidar, and only retain the point cloud in front.

[0168] (3) Filter out false points generated by lidar noise.

[0169] By filtering out invalid points using the above preprocessing method, we can reduce the amount of computation and avoid invalid points interfering with the subsequent cliff detection algorithm and causing misjudgments.

[0170] Step 412: Construct a two-dimensional matrix diagram based on the three-dimensional point cloud data.

[0171] The three-dimensional point cloud data obtained in step 411 contains the following information: the three-dimensional spatial coordinates (x, y, z) of each laser point and the row and column index (row, col) of the lidar. Based on the three-dimensional spatial coordinates (x, y, z), the horizontal distance (dis) and height information (z) of each laser point from the lidar can be determined.

[0172] Initialize the distance matrix (dis_map) and the height matrix (z_map); the distance matrix (dis_map) is used to represent the distance information (dis) of each laser point, and the height matrix (z_map) is used to represent the height information (z) of each laser point.

[0173] Traverse all laser points in the 3D point cloud data, and based on the row and column index (row, col) of each laser point, perform the following two-dimensional matrix assignment operations on the distance matrix (dis_map) and height matrix (z_map): assign the horizontal distance (dis) of the current laser point to the corresponding row and column position in the distance matrix (dis_map); assign the height value (z) of the current laser point to the corresponding row and column position in the height matrix (z_map).

[0174] The above assignment achieves dimensionality reduction mapping from 3D laser point cloud to 2D raster space, transforming disordered 3D point cloud data into two ordered and regular 2D raster matrix images. Subsequent steps only require traversing these two 2D raster matrix images in row and column order to obtain cliff candidate points. The algorithm logic is simple, consumes less memory, and speeds up computation.

[0175] However, it should be noted that while using a two-dimensional raster matrix can speed up the operation, the three-dimensional spatial information is lost during the rasterization process. The distance and height information stored in the raster matrix cannot support subsequent advanced functions such as fine-grained geometric verification and path planning. Therefore, it is necessary to construct a hash mapping table (point_map) from the row and column indices (row,col) of the two-dimensional raster matrix to the original three-dimensional point cloud. This allows the cliff candidate points that pass the initial screening in the two-dimensional raster matrix to recover their complete three-dimensional coordinate information, thus balancing detection efficiency and verification accuracy.

[0176] The specific steps for constructing a hash mapping table from the row and column indices (row, col) of a 2D raster matrix to the original 3D point cloud are as follows: Using the row and column indices (row, col) as the key and the spatial coordinates as the value, a mapping entry from the 2D index to the original point cloud data is established. Through this hash mapping table, in subsequent processing, the corresponding original 3D point cloud data can be directly accessed based on any row and column index (row, col), avoiding information loss due to rasterization dimensionality reduction.

[0177] Step 413: Detect cliff feature points based on the two-dimensional matrix diagram.

[0178] In some embodiments, cliffs are detected in real time as the mobile device moves throughout the work area.

[0179] In other embodiments, because there are some passable gentle slopes within the work area, when the self-moving device prepares to descend a slope, the lidar's downward detection angle is similar to the slope angle, causing it to be mistakenly detected as a cliff. This prevents the self-moving device from advancing smoothly down the slope, reducing work efficiency. Therefore, this embodiment does not execute cliff detection logic throughout the entire work area, but instead sets trigger conditions for cliff detection. Cliff detection logic is only activated when the self-moving device is near the boundary (i.e., when the self-moving device is less than or equal to a preset value from the boundary of the work area). The preset value is set by those skilled in the art based on experience, for example, 1m, and is not limited here.

[0180] It's easy to understand that to ensure the self-moving device can move smoothly within the work area, a boundary map of the work area must be established. In this embodiment, it is assumed that the boundary map is a raster map. The method for detecting the distance between the self-moving device and the boundary of the work area is as follows:

[0181] The system determines the current location of the self-moving device. Centering this location, a 2m x 2m local area is defined. All grid cells within this local area are traversed. It checks if any grid cell within the local area has a boundary attribute. If such a grid cell exists, the distance between the self-moving device and the boundary of the working area is determined to be less than a preset value, thus meeting the cliff detection trigger condition. If no grid cell within the local area has a boundary attribute, the distance between the self-moving device and the boundary of the working area is determined to be too great, and the area near the self-moving device is considered open; therefore, cliff detection is not performed.

[0182] By setting the above triggering conditions, false cliff signals caused by passable gentle slopes, grass clippings, or sensor noise within the work area are effectively filtered out, preventing the mobile equipment from frequently stopping or detouring due to misjudgment, thus improving the continuity and coverage efficiency of the operation.

[0183] When the distance between the self-moving device and the boundary of the working area is less than or equal to a preset value, the specific details of performing cliff detection are as follows:

[0184] The distance matrix (dis_map) and height matrix (z_map) are traversed grid by grid, and each grid is determined to be a valid grid point that meets the preset conditions.

[0185] If a grid point is a valid grid point, local feature calculations are performed on that grid point to determine whether it is a cliff feature (edge) point; if a grid point is not a valid grid point, it is directly filtered without determining whether it is a cliff point, thereby reducing the amount of computation and speeding up the detection.

[0186] Grid points in the following three cases are considered invalid and are directly filtered out:

[0187] (1) If the height value z>0 of a certain grid point in the height matrix map (z_map), it indicates that the laser point corresponding to the grid point is reflected by an object higher than the lidar. This object is likely a protruding obstacle, which is significantly different from the cliff feature. Therefore, the grid point is filtered out and there is no need to detect whether the grid attribute is a cliff.

[0188] (2) If the horizontal distance value (dis) of a grid point in the distance matrix (dis_map) is greater than the preset value (e.g., 1m), it indicates that the laser point corresponding to the grid point is reflected by an object located more than 1m in front of the lidar. The distance to the self-moving device is far and has no impact on the current operation of the self-moving device. Therefore, no judgment is made for the time being, and the grid point is also filtered out.

[0189] (3) Traverse each grid point in order from near to far. If the number of grid points that have been identified as cliff feature points in a certain column (col) is greater than the preset value, it is determined that the location of the column is likely to be a cliff. The mobile device will avoid the location of the column in the future. The subsequent distant grid points in the column have no impact on the path planning of the mobile device. The detection of the subsequent distant grid points in the column is directly terminated to further reduce the amount of computation and ensure real-time performance.

[0190] The specific method for performing local feature calculations on effective grid points to determine whether a valid grid point is a cliff feature point is as follows:

[0191] Centered on the current valid grid point, sampling is performed in the neighborhood in four directions: up, down, left, and right. If the neighboring grid point is invalid, the search continues for valid points by extending 1-2 grids along the original direction to ensure the integrity of local feature calculation. If no valid points are found after extending 1-2 grids, the sampling data in that direction is discarded to avoid invalid data interfering with the judgment result.

[0192] After sampling is completed in the neighborhood in the four directions of up, down, left, and right, the height value (z) and horizontal distance value (dis) of each sampling point in the neighborhood are determined, and the maximum height value z_max, minimum height value z_min, maximum horizontal distance value dis_max, and minimum horizontal distance value dis_min in the neighborhood are obtained by comparison.

[0193] Determine the height (z) and horizontal distance (dis) of the current valid grid point;

[0194] If the current grid point's height (z) and horizontal distance (dis) satisfy any of the following conditions, then the current grid point is determined to be a cliff feature (edge) point:

[0195] Condition 1: The difference between the current grid point's height value (z) and the maximum height value z_max is greater than a first preset value. In this case, the current point is located at the bottom of a cliff, while the neighborhood includes points at the top of the cliff; for example, the current point is located in a pit or depression.

[0196] Condition 2: The difference between the current grid point's height value (z) and the minimum height value z_min is greater than a first preset value; in this case, the current point is located at the top of a cliff, while the neighborhood includes points at the bottom of the cliff; for example, the current point is located at the upper edge of a protrusion or step.

[0197] Condition 3: The difference between the horizontal distance value (dis) of the current grid point and the maximum horizontal distance value dis_max is greater than the second preset value. In this case, the current point is located at the edge of a nearby object, while the neighborhood includes the distant background (such as a distant wall), resulting in a ground discontinuity between the nearby object and the distant wall.

[0198] Condition 4: The difference between the horizontal distance value (dis) of the current grid point and the minimum horizontal distance value dis_min is greater than the second preset value. In this case, the current point is located in a distant background (such as a distant wall), while the neighborhood contains nearby objects; the current point is located behind a ground fault, for example, the current point is reflected from a wall behind a ground fault.

[0199] The first and second preset values ​​mentioned in conditions 1-4 are set by those skilled in the art based on experience. For example, the first preset value is 0.1m and the second preset value is 1m. No limitation is made here.

[0200] It is easy to understand that on a flat surface, the height and horizontal distance values ​​between adjacent grid points should transition smoothly. When a cliff edge appears, a significant numerical abrupt change occurs within the local window. This embodiment identifies ground faults (cliffs) by analyzing height or distance abrupt changes in the local neighborhood, transforming the "spatial discontinuity detection" problem into a "comparison of local extreme value differences with a threshold" problem, thus achieving efficient, accurate, and robust cliff edge detection.

[0201] Step 414 generates a three-dimensional cliff point cloud based on the cliff feature points.

[0202] The cliff feature (edge) points obtained through step 413 above are two-dimensional grid points. Each grid position retains only the horizontal distance and height values; the original three-dimensional coordinates (X, Y, Z) are discarded. The precise spatial location of edge points cannot be obtained through discrete grid points in a two-dimensional grid, making it impossible to directly fit a continuous cliff boundary line or calculate the cliff's depth and width. Furthermore, although some grid points are identified as cliff feature points at the grid level, these grids often exhibit isolated and discrete characteristics in three-dimensional space, which does not conform to the continuous linear distribution of a real cliff.

[0203] Therefore, it is necessary to reconstruct the cliff feature points from two dimensions into a three-dimensional cliff point cloud to further verify the accuracy of cliff detection and obtain more precise cliff information. Specific details are as follows:

[0204] Obtain the binary edge mask matrix edge_mask generated in step 413 above. In edge_mask, each grid cell has a value of 1, which indicates that the position is a cliff feature (edge) point, and a value of 0 indicates that the position is a non-edge point.

[0205] Iterate through all grid points in edge_mask. For each grid point, determine whether its mask value is 1. If it is 1, use the row and column index (row,col) of the current grid point as the query key to retrieve the hash map table point_map pre-built in step 412 to obtain the original laser point 3D coordinates (x,y,z) corresponding to the grid point, and store the coordinates in the point cloud container to form the initial 3D cliff point cloud set cloud_initial.

[0206] However, the initial 3D cliff point cloud (cloud_initial) has a high error rate, containing a mixture of various data types, including real cliff edges, non-edge point clusters, blocky obstacle points (such as rocks and tree stumps), and noise points (such as flying points, ghosting, and rain / fog reflections). Therefore, further filtering of the initial 3D cliff point cloud (cloud_initial) is needed to obtain the final 3D cliff point cloud data, which only contains real cliff edge data.

[0207] Specifically, the manner of further screening the preliminary 3D cliff point cloud cloud_initial to obtain the final 3D cliff point cloud data includes the following steps:

[0208] (1) Euclidean clustering, which is configured to filter out mixed isolated noise points and flying points in the preliminary 3D cliff point cloud cloud_initial, and distinguish continuous edges from discrete noise. The specific process is as follows:

[0209] Set a clustering search radius (used to determine whether two points belong to the same clustering cluster) and a minimum cluster point number threshold N_min (for example, 5 to 15 points, used to distinguish a valid edge cluster from an isolated noise point);

[0210] Traverse each laser point in the preliminary 3D cliff point cloud cloud_initial, take the point as a seed point, search for neighboring points within the clustering search radius, and perform recursive expansion to form a clustering cluster;

[0211] Count the number of points N_cluster contained in each clustering cluster;

[0212] If N_cluster < N_min, the cluster is determined to be an isolated noise point or scattered flying point, and is eliminated;

[0213] If N_cluster ≥ N_min, the cluster is retained, and the process proceeds to the next step.

[0214] (2) Linear geometric feature screening: eliminating planar structural obstacles to ensure that only the real cliff edges extending in a straight line are retained.

[0215] Since the point cloud of the real cliff feature (edge) is approximately linearly distributed in space, its geometric feature is that the maximum eigenvalue λ1 is significantly larger than the other two eigenvalues (λ2 and λ3), that is, the point cloud mainly extends along a single direction. The linear extension degree of the clustering cluster in space can be quantified by calculating the linearity of the clustering cluster. The closer the linearity value is to 1, the more obvious the extension of the point cloud cluster in the direction corresponding to the maximum eigenvalue, and the weaker the extension in the direction corresponding to the intermediate eigenvalue, that is, the closer the point cluster is to "linear" (such as a cliff edge, which is mostly linearly distributed); the closer the value is to 0, the closer the point cluster is to "planar" or "spherical" (non-cliff edge feature). Based on this, whether the above clustering cluster conforms to the cliff edge feature can be judged according to the linearity value.

[0216] Specifically:

[0217] Assume that the clustering cluster satisfying the condition in the above step (1) is {p i |p i =(x i ,y i ,z i ) T,i=1,2,…,N;

[0218] Calculate its centroid μ and covariance matrix S. The covariance matrix characterizes the dispersion of the point cloud distribution in the three spatial dimensions. The calculation formula is as follows:

[0219] Where p i The three-dimensional coordinates of each laser point in the cluster are represented by N, and N represents the total number of laser points in the cluster.

[0220] The formula for calculating linearity L is as follows:

[0221] Where λ1, λ2, and λ3 are the three eigenvalues ​​obtained by eigenvalue decomposition of the covariance matrix S. λ1 is the smallest eigenvalue, corresponding to the direction in which the cluster extends the shortest distance in space; λ2 is the middle eigenvalue, corresponding to the direction in which the cluster extends the middle distance in space; and λ3 is the largest eigenvalue, corresponding to the direction in which the cluster extends the longest distance in space. ε is a local minimum value used to prevent the denominator from being zero and to avoid numerical calculation errors. In application, for example, ε is set to 10. -6 10 -8 wait.

[0222] Set minimum linearity threshold L min If the linearity L of the current cluster is less than L min This indicates that it does not conform to the linear characteristics of the cliff edge and is therefore eliminated;

[0223] If the linearity L of the current cluster is greater than or equal to L min This indicates that the cluster conforms to linear geometric characteristics, and it is determined to be a true cliff edge cluster, which is retained.

[0224] (3) Uniform downsampling: While preserving the geometric features of the cliff edge, the point cloud size is compressed to a preset range to adapt to the computing power limitations of the embedded system. Specifically:

[0225] Set the target sampling interval, perform voxel downsampling or uniform grid sampling on the point cloud in the real cliff edge cluster obtained in step (2), and retain only one representative point (such as the centroid or the point closest to the voxel center) in each voxel grid;

[0226] All downsampled valid points are merged to generate the final 3D cliff point cloud, cloud_final, which can be directly used by the downstream path planning module.

[0227] Based on the final 3D cliff point cloud (cloud_final), the path planning module of the self-moving device can obtain the precise location, geometry, and width information of the cliff edge. Based on this, the path planning module no longer relies solely on the simple "cliff present / absent" signal, but can perform refined and intelligent motion control based on the specific parameters of the cliff, significantly improving the operating efficiency and environmental adaptability of the self-moving device while ensuring safety.

[0228] The above embodiments use a three-stage processing approach of "clustering denoising, linear screening, and downsampling compression" to transform coarse candidate cliff point clouds into accurate, lightweight, and standardized cliff feature point clouds, balancing detection accuracy and system real-time performance.

[0229] The cliff detection method provided in this application first rasterizes (reduces dimensionality) the 3D point cloud data to obtain a 2D raster matrix. The 2D raster matrix is ​​then traversed point by point, and points satisfying height / distance abrupt change conditions are identified as cliff feature points. These detected 2D cliff feature points are then reconstructed into a preliminary 3D cliff point cloud for further fine-tuning, resulting in the final 3D cliff point cloud data. Rasterizing (reduces dimensionality) the 3D point cloud data into a 2D raster matrix reduces computation and speeds up detection; reconstructing the 2D cliff feature points into a 3D cliff point cloud improves accuracy and obtains more precise cliff information. The combination of these two methods achieves an optimized architecture of "rapid initial screening + precise verification," enabling fast and accurate cliff detection that meets the real-time operation requirements of mobile devices. Furthermore, this application embodiment only requires 3D point cloud data to achieve fast and accurate cliff detection, without the need to integrate other sensor data (such as visual sensor data), thus avoiding additional hardware costs.

[0230] Please refer to Figure 6. Taking the aforementioned self-moving device as the execution subject, this application embodiment also provides a control method for the self-moving device. The self-moving device is equipped with a non-scanning single-frame pulse imaging sensor. The self-moving device moves within the working area. The method includes the following steps 421-422.

[0231] Step 421: Acquire three-dimensional point cloud data using a non-scanning single-frame pulse imaging sensor.

[0232] Step 422: When the distance between the self-moving device and the boundary of the working area is less than or equal to a preset distance threshold, cliff detection is performed based on the three-dimensional point cloud data, and the movement strategy of the self-moving device is determined based on the cliff detection result.

[0233] It's easy to understand that the work area contains a large number of natural terrain features with similar point cloud characteristics to cliffs (such as passable downhill slopes and low-lying areas). If cliff detection is enabled across the entire area, the mobile device will also identify such passable areas as cliffs and then perform preset operations to avoid these passable areas, which undoubtedly seriously affects the working efficiency of the mobile device.

[0234] The boundaries of the work area often have abrupt changes in terrain (e.g., grassland → paved road, grassland → steps, grassland → water). There is a lack of transition areas at these abrupt changes. Once the mobile equipment crosses the boundary, it loses support and faces the risk of falling off the boundary, which may lead to equipment damage, personnel injury and other hidden dangers.

[0235] Therefore, there is a high probability that there is a cliff at the boundary of the work area, and the probability of the self-moving device falling off the cliff at the boundary of the work area is even greater. Therefore, accurately detecting cliffs near the boundary and taking corresponding control measures is a prerequisite for ensuring the safe operation of self-moving devices.

[0236] For details on performing cliff detection based on 3D point cloud data, please refer to the cliff detection method for self-moving devices mentioned above, which will not be repeated here.

[0237] Once a cliff is detected, the movement strategy of the self-moving device is determined based on the cliff detection results, specifically including the following details:

[0238] If a cliff is detected, and the distance between the self-moving device and the cliff is less than a preset distance threshold, then the self-moving device is controlled to stop moving or turn.

[0239] If a cliff is detected and the distance between the self-moving device and the cliff is greater than a preset distance threshold, the self-moving device is first controlled to slow down and move forward. When the distance between the self-moving device and the cliff is less than a first preset distance, the self-moving device is then controlled to stop moving or turn.

[0240] The control method provided in this embodiment triggers cliff detection only when the self-moving device approaches the boundary of the working area, achieving differentiated control of "efficient operation in the internal area and precise protection in the boundary area." Compared with a continuous detection scheme for the entire area, it significantly reduces system power consumption and false trigger rate within the area while ensuring safety. Furthermore, the method provided in this embodiment adopts a graded and scenario-based differentiated control strategy based on the distance to the cliff, significantly improving the working efficiency and intelligence level of the self-moving device while ensuring safety.

[0241] In related technologies, self-moving devices, such as automatic lawnmowers, use LiDAR sensors to scan their surroundings and process the scanned data to identify the location, shape, and size of surrounding objects (such as trees, flower beds, and walls). Based on the identified objects, the automatic lawnmower can ultimately achieve autonomous navigation, obstacle avoidance, and precise mowing. However, LiDAR obstacle detection solutions have a significant drawback: ramp misjudgment. Traditional methods filter point clouds using only a single height threshold, failing to distinguish between sloping ramps and vertical obstacles, leading to ramp point clouds being misjudged as obstacles and limiting the machine's range of motion. For example, relying on a separate AI module for ramp filtering requires significant computational resources, has poor real-time performance, and transmits a large amount of point cloud data containing noise and ramp information to subsequent modules (such as SLAM / path planning), increasing the downstream burden. If the AI ​​module is removed and ramp filtering is handled solely by SLAM, accuracy and real-time performance are difficult to guarantee. Furthermore, sparse noise points interfere with detection accuracy; fixed thresholds are difficult to adapt to ramp scenarios with different tilt angles and distances, resulting in poor generalization ability.

[0242] To address the aforementioned issues, in some embodiments, step S102 includes: identifying a ramp based on three-dimensional point cloud data according to a fifth preset condition; wherein the fifth preset condition includes at least one of the following: point height condition, Euclidean distance condition between adjacent points, point cloud direction abrupt change condition, and tilt angle condition of point cloud extension direction.

[0243] The inventors discovered that the height of a point, the Euclidean distance between adjacent points, abrupt changes in the point cloud direction, and the tilt angle of the point cloud extension direction can all characterize a ramp to some extent. It can be understood that this embodiment identifies ramps based on one or more of the following conditions: point height, Euclidean distance between adjacent points, abrupt changes in the point cloud direction, and the tilt angle of the point cloud extension direction. This adapts to the real-time navigation needs of mobile devices, automatically identifying and filtering various types of ramp point clouds (adapting to ramp scenarios with different tilt angles and distances), retaining only the point clouds of real obstacles, with no obvious misjudgments or omissions, thus improving detection accuracy.

[0244] Please refer to Figure 7. This embodiment provides a control method for a self-moving device, applied to identify target objects (such as ramps, obstacles, etc.) in the working environment of the self-moving device, including the following steps:

[0245] Step 521: Obtain at least one frame of 3D point cloud data from the target scene.

[0246] The 3D point cloud data is generated by a non-scanning single-frame pulse imaging sensor scanning the target area according to a preset scanning angle and scanning sequence. The sequence of the 3D point cloud data corresponds to the physical scanning path of the non-scanning single-frame pulse imaging sensor. When the mobile device enters the target area, the non-scanning single-frame pulse imaging sensor scans the target area to obtain the 3D point cloud data.

[0247] The target area may include target marker objects, which may include pre-set reflectors, objects in the preset area, obstacles in the driving path, or slopes or depressions in the driving path.

[0248] The initial data collection will first determine whether it is reliable before processing. For example, it will consider whether the driving status of the integrated equipment is stable, road condition information, and the total number of points returned by the sensors to determine whether the acquired 3D point cloud data is reliable.

[0249] After acquiring multiple sets of continuous frames of 3D point cloud data, clustering filtering is performed on the 3D point cloud data to reduce outliers. The 3D point cloud data is then projected based on the pitch and roll angles in the equipment's operating environment to ensure the reliability of the 3D point cloud data.

[0250] Step 522: According to the first screening rule, perform preliminary screening on the current frame 3D point cloud data to determine the first type of point cloud data and the second type of point cloud data in the current frame;

[0251] Based on the grid ground height of the target area and the preset obstacle height threshold, the single-frame point cloud filtered out in step 521 is traversed and judged point by point to achieve preliminary screening of passable areas, providing target data for the segmented feature analysis in step 523.

[0252] The first filtering rule is based on a pre-established and dynamically maintained two-dimensional grid ground height reference. It compares the height of the current point cloud with the ground height reference of its grid and makes a judgment based on an adaptive obstacle height threshold. The adaptive obstacle height threshold is related to the grid size and the maximum passable slope of the device.

[0253] For example, for each point in the point cloud data, perform the following filtering logic:

[0254] Based on the horizontal coordinates of the point, determine its corresponding 2D grid and query the corrected dynamic ground height value of that grid. Calculate the difference between the actual height value of the point and the corresponding grid's dynamic ground height value. Compare this difference with an adaptive obstacle height threshold. This threshold is not a fixed value but is dynamically determined by considering factors such as grid physical size, maximum traversable slope angle of the device, and point cloud error tolerance to ensure that point clouds on the maximum traversable slope are not misjudged. If the difference is greater than the adaptive obstacle height threshold, the point is determined to be first-type point cloud data (e.g., obstacle or impassable area) and directly output to the obstacle result set; otherwise, the point is determined to be second-type point cloud data (e.g., suspected slope point or traversable area).

[0255] The first type of point cloud data is used to indicate clear obstacles or impassable areas beyond the equipment's capabilities (such as walls, trees, large rocks, etc.). This type of data will be directly output as the final obstacle information and will not participate in subsequent processes. The first type of point cloud data refers to the set of point clouds with significant elevation anomalies directly confirmed by the height difference determination logic in this step.

[0256] The second type of point cloud data refers to point cloud sets that have not passed the height difference determination. It indicates potential terrain requiring further detailed geometric feature analysis to determine its traversability. This type of data may include continuous point clouds forming ramps, but may also contain low-lying or real obstacles with specific angles. This data is labeled as "potentially traversable areas" and serves as input for subsequent segmentation and feature analysis.

[0257] The first type of point cloud data is used to indicate impassable areas; the second type of point cloud data is used to indicate potentially passable areas. Alternatively, the second type of point cloud data needs to be further analyzed through the second filtering rules and subsequent steps to ultimately determine the passability of the indicated area.

[0258] Step 523: According to the second filtering rule, process the second type of point cloud data in segments to generate multiple segments of point cloud data with spatial continuity;

[0259] According to the second screening rule, the second type of point cloud data output in step 522 is processed online and intelligently clustered into multiple spatially continuous point cloud segments, with each segment serving as an independent analysis unit.

[0260] The second filtering rule triggers segmentation based on multi-dimensional spatial continuity measurement and scene semantic interruption conditions between point cloud data; wherein, the spatial continuity measurement includes at least distance continuity reflecting location proximity and directional continuity reflecting the consistency of extension trend; the scene semantic interruption condition refers to the current point cloud being determined by the first filtering rule to be point cloud data of the first type.

[0261] The second filtering rule indicates that dynamic segmentation should be performed based on the native spatial relationships of the point cloud data. This rule includes at least two types of criteria:

[0262] Spatial continuity criterion: based on the inherent correlation between scanning time sequence and spatial location of 3D point cloud data;

[0263] Segmentation interruption criterion: Segmentation is triggered immediately when the input current point cloud is identified as Type I point cloud data. This ensures the isolation of obstacle point clouds and ramp point clouds on the analysis unit.

[0264] For example, the first type of point cloud data is processed, and a current point cloud sequence is dynamically maintained. For each new input point, its continuity index with the current sequence is calculated in real time, and its type is determined. Once any of the above criteria is met, the current sequence is determined to have ended, it is output as an independent point cloud segment, and the sequence is cleared to begin the accumulation of the next segment.

[0265] The first type of point cloud data is segmented using the second filtering rule. Point clouds with the same spatial correlation are divided into the same valid segment, providing an independent analysis unit for subsequent angle calculation and slope determination. This ensures that each segment of point cloud belongs to the same obstacle or the same continuous slope, fundamentally avoiding the mixing of point clouds of different objects or different slope areas for analysis. This is a key preprocessing step to improve the accuracy of subsequent geometric feature extraction and the reliability of slope determination.

[0266] Step 524: Extract the spatial characteristics of each segment of point cloud data and determine the slope angle of each segment of point cloud data based on the target ground.

[0267] This step is used to extract core geometric features representing the terrain orientation from the segmented continuous point cloud. By quantifying the overall tilt trend of the point cloud segment through the main extension direction, the geometric distribution of the three-dimensional point cloud is transformed into a slope angle that can be directly used for passage judgment, realizing the quantitative conversion from "point cloud structure" to "slope value".

[0268] For example, the target ground mentioned above can be the original ground in the target area, or it can be a ground modified based on the original ground in the target area.

[0269] For example, based on the overall distribution of point cloud segments, their main spatial extension direction is extracted. This direction reflects the longest extension trend of the point cloud segment in three-dimensional space and corresponds to the direction of the terrain slope. The angle between the main extension direction and the vertical axis of the world coordinate system is calculated, and the resulting angle is used to quantitatively characterize the degree of inclination of the ground corresponding to the point cloud segment.

[0270] Step 525: If the slope angle of the current segment of point cloud data is greater than or equal to the preset value, then mark the current segment of point cloud data as the first type of point cloud data.

[0271] This step, based on the slope angle extracted and calculated in the previous steps, combined with the preset slope threshold set by the device's passability capability, completes the passability classification and determination of the terrain corresponding to the point cloud segment. This enables the automatic filtering of passable slopes and the retention and output of impassable terrain (steep slopes / obstacles), providing direct and reliable determination results for the path planning and obstacle avoidance decision of the self-moving device.

[0272] The preset slope threshold is set in advance based on the mechanical structure, driving ability and operational safety requirements of the self-moving device, and represents the maximum safe slope that the device is allowed to pass. The slope angle directly reflects the overall tilt of the terrain corresponding to the point cloud segment: the larger the slope angle, the steeper the terrain; the smaller the slope angle, the gentler the terrain.

[0273] The above determination results correspond one-to-one with the scanning columns of the non-scanning single-frame pulse imaging sensor, and can be directly mapped to the local path in front of / around the device, realizing column-by-column, segment-by-segment, and refined path marking.

[0274] If the slope angle is greater than or equal to the preset slope threshold, the terrain corresponding to the point cloud segment is determined to be an impassable area, and the area corresponding to the point cloud segment is marked as an impassable area.

[0275] If the slope angle is less than the preset slope threshold, the terrain corresponding to the point cloud segment is determined to be a passable area, which facilitates subsequent path planning.

[0276] If the slope angle is less than the first preset slope threshold, the terrain corresponding to the point cloud segment is determined to be a flat area, and the current driving strategy continues to be executed; if the slope angle is less than the second preset slope threshold, the terrain corresponding to the point cloud segment is determined to be a slope area, and the current driving strategy is switched to the slope driving strategy.

[0277] For example, if the slope angle of a certain point cloud segment is determined to be greater than the preset maximum passable slope threshold (this threshold can be set according to the device's climbing ability, for example, 70°), then the area represented by that point cloud segment will be determined to be an impassable steep slope or obstacle. This point cloud segment will be marked as first-type point cloud data (i.e., obstacle) and output to the obstacle result set for subsequent obstacle avoidance processing.

[0278] Subsequently, the self-moving device will treat this area the same as other obstacles, using a detour or avoidance strategy to ensure the safety of the device and the continuity of operations.

[0279] Based on the analyzed point cloud data, machine learning or image processing techniques can be used to identify target objects (such as automatic lawnmowers identifying lawn boundaries, obstacles, etc.), and the movement path of the self-moving device can be adjusted according to the identification results to achieve precise positioning and operation.

[0280] The method provided in this application captures three-dimensional point cloud data using a non-scanning single-frame pulse imaging sensor, and combines layered filtering processing with dynamic segmentation to effectively avoid the slope point cloud being misjudged as an obstacle, thus solving the problem of poor adaptability of slopes with different gradients in the prior art.

[0281] The method provided in this application, through point cloud data collection, initial screening and classification, spatial continuity segmentation, spatial main direction feature extraction, slope angle determination and accessibility marking, achieves accurate differentiation between obstacles and ramps. Without relying on AI modules, it significantly improves terrain recognition accuracy, system real-time performance and environmental robustness, and is particularly suitable for autonomous obstacle avoidance and safe operation of self-moving devices such as automatic lawnmowers in complex terrain.

[0282] In an optional embodiment, the method provided in this application can also perform filtering processing after collecting point cloud data in step 521.

[0283] Taking an automatic lawnmower as an example, this lawnmower is equipped with a high-performance non-scanning single-frame pulse imaging sensor, such as a solid-state area array LiDAR. After startup, it automatically enters a preset lawn area to scan. The LiDAR collects three-dimensional data of the surrounding environment at a rate of tens of thousands of points per second. After receiving this data, the data processing unit performs point cloud extraction and filtering to remove noise points, isolated points, and redundant information unrelated to the target operation scene (lawn, slope, obstacles). At the same time, it retains as many key structural features of the ground, slope, and obstacles as possible, providing clean point cloud data for subsequent rasterization processing and obstacle identification, thereby improving the processing accuracy and efficiency of subsequent steps.

[0284] In an optional embodiment, step 522 of the method provided in this application, which determines the first type of point cloud data and the second type of point cloud data in the current frame, includes: steps 531 to 533:

[0285] Step 531. After scanning the target area according to the preset rules, obtain the 3D point cloud data of the target area and generate a 2D raster map of the target area;

[0286] Step 532. According to the preset height constraint, the initial ground height of the target area is corrected to generate the target ground height of the target area, so that the first filtering rule is executed according to the target ground height;

[0287] The preset height constraints may include at least one of the following: the physical parameters of the equipment and preset height data, and the slope of the working area ground. For example, the set of physical parameters of the equipment may include at least the installation height of the non-scanning single-frame pulse imaging sensor, the sensor field of view, and the ground clearance of the equipment chassis.

[0288] Of course, the above conditions are not static thresholds, but rather embed the device's inherent attributes and prior environmental knowledge into the ground model construction process to solve the ground perception distortion problem caused by near-field occlusion, blind spot interference, and fuselage reflection.

[0289] For example, the step 532 above, which involves correcting the initial ground elevation of the target area, includes:

[0290] The point cloud's XY plane is divided into multiple 2D grids according to a preset grid size using a hash table. All filtered point clouds are traversed, and the minimum Z-value of all point clouds within each grid (i.e., the initial ground height corresponding to that grid) is recorded, thus establishing a preliminary correspondence between the grid and the ground height. The preset grid size can be set according to the sensor's accuracy and the operational scenario, such as 0.1m × 0.1m.

[0291] For the grid corresponding to the sensor's near-field region (e.g., the near-field region is the range close to the self-moving device, such as 0-1m, set according to the device size), ground height correction is performed: if the minimum Z value recorded during the initialization of a certain near-field grid is higher than the radar installation height, it is determined that the ground corresponding to that grid is occluded (e.g., by the vehicle body or low near-field obstacles). At this time, the ground height of that grid is forcibly corrected to the preset ground height threshold to avoid ground reference distortion caused by near-field point cloud noise and ensure the accuracy of obstacle judgment in the near-field region.

[0292] By combining factors such as the sensor's installation height, the equipment's operating scenario, or user-defined settings, a unified ground height threshold is defined for the minimum Z value of all grids. This avoids ground height benchmark inconsistencies caused by differences in different regional scenarios (e.g., near-field occlusion, far-field ground undulations), ensuring consistent reference standards for subsequent obstacle screening.

[0293] The method provided in this application performs rasterization processing before classifying point cloud data and simultaneously corrects ground height, thereby confirming a unified and accurate ground height benchmark. This provides a reliable reference standard for subsequent initial screening of point cloud data, reduces slope misjudgment and obstacle omission due to benchmark deviation, and solves the problem of slope misjudgment caused by ground benchmark distortion.

[0294] In an optional embodiment, step 522 of the method provided in this application, which determines the first type of point cloud data and the second type of point cloud data in the current frame, further includes:

[0295] Step 541. Traverse each point cloud data in the target frame and query the target ground height value of the raster where each point cloud data is located;

[0296] Step 542. Calculate the difference between the current point cloud data height value and the ground height value;

[0297] Determine whether the difference is greater than a preset obstacle height threshold; wherein, the adaptive obstacle height threshold is calculated based on the grid size and the maximum passable slope angle of the mobile device;

[0298] Step 543. If the difference is greater than the preset obstacle height threshold, then the current point is determined to be a point cloud of the first type of obstacle.

[0299] The preset height threshold is calculated based on the grid size and the maximum passable slope angle of the mobile device.

[0300] The method provided in this application embodiment achieves efficient initial screening through a quantified height difference threshold, reducing the amount of data in subsequent segmented processing and improving overall computational efficiency. At the same time, the aforementioned height difference threshold is calculated from the physical parameters of the device and the requirements of the scenario, ensuring the universality and accuracy of the rules and preventing gentle slopes from being misjudged as obstacles.

[0301] In an optional embodiment, step 202 of the method provided in this application, which determines the first type of point cloud data and the second type of point cloud data in the current frame, further includes:

[0302] Step 551. Generate the first point cloud sequence based on the second type of point cloud data.

[0303] For example, the scanned temporal point cloud data P0 to Pn are stored in the first point cloud sequence according to the spatial sequence;

[0304] Step 552. Calculate the spatial continuity measure between two adjacent points in the first point cloud sequence;

[0305] Step 553. Based on the spatial continuity measure between adjacent point clouds in the first point cloud sequence, determine whether the current point cloud meets the segmentation condition;

[0306] Step 554. If the current point cloud in the first point cloud sequence meets the segmentation condition, then store the current point cloud data into the second point cloud sequence to generate multiple valid point cloud data segments.

[0307] The spatial continuity metric is a quantitative indicator that characterizes the strength of the geometric association between adjacent point clouds in three-dimensional space. Spatial continuity metrics can include dimensions such as positional continuity, directional continuity, or density continuity.

[0308] For example, the spatial continuity measure is used to measure the strength of association between adjacent points in a first sequence to determine whether they belong to the same continuous entity, thereby enabling the aggregation of continuous point clouds that should belong to the same physical surface (such as a slope) while separating point clouds that belong to different objects or discontinuous parts.

[0309] For example, in the segmentation process, this application embodiment can simultaneously maintain two queues: a point cloud queue (which stores the suspected ramp points to be segmented) and a point cloud direction queue (which stores the direction vectors formed by adjacent points in the current point cloud segment, used for segmentation condition judgment), to ensure the continuity of point-by-point processing and the accuracy of segmentation.

[0310] For example, the segmentation conditions in step 553 above may include:

[0311] Condition A: Based on distance features, the Euclidean distance between the new point cloud and the previous point cloud in the sequence exceeds a preset threshold;

[0312] Condition B: The new point cloud is detected to be a first type of point cloud;

[0313] Condition C: Based on directional features, the angle between the directional trend determined by the current sequence of historical point clouds and the directional vector formed by the new point cloud and the previous point exceeds a preset threshold.

[0314] If any triggering condition is met, the current sequence will be output as a complete point cloud segment.

[0315] In condition C, the point cloud direction changes abruptly. The judgment logic is as follows: ① Calculate the mean of the historical direction vectors in the point cloud direction queue (historical direction, i.e., the average of the direction vectors formed by the previous n adjacent points, where n is set according to the point cloud frame rate); ② Calculate the current direction vector formed by the current input point and the previous point; ③ Normalize the mean of the historical direction vectors and the current direction vector, and calculate the angle between them. If the angle exceeds the preset direction threshold, it is judged as a direction change, and segmentation is triggered.

[0316] For example, after determining whether the current point cloud meets the segmentation conditions in step 553 of the above steps, the process includes:

[0317] Case 1: If the input point is a suspected ramp point and the point cloud queue is empty (the first suspected ramp point), directly add the point to the point cloud queue, and at the same time calculate the direction vector between the point and the subsequent input points (triggered when the subsequent points are input), and add it to the point cloud direction queue.

[0318] Case 2: If the input point is a suspected ramp point and the point cloud queue is not empty (not the first point), check whether the above segmentation triggering conditions are met. If any triggering condition is met, the point cloud in the current point cloud queue is taken as a valid segment and output to step six for subsequent analysis. At the same time, the point cloud queue and the point cloud direction queue are cleared, and the current input point is added to a new point cloud queue. If the triggering condition is not met, the current point is added to the point cloud queue, the direction vector between the current point and the previous point is calculated, and added to the point cloud direction queue.

[0319] Case 3: The input point is the last point of all point clouds to be processed in the current frame, and the point cloud queue is not empty (to avoid missing the last segment of suspected ramp points in a frame). Perform segmentation judgment on the current point cloud queue. If no segmentation is required, put the point into the point cloud queue and output it as a valid segment to step six. If segmentation is required, output the point cloud queue directly as a valid segment. After processing, clear both queues and the point cloud processing of the current frame ends.

[0320] The method provided in this application relies on the original spatial continuity of point clouds and uses a multi-condition triggering mechanism to achieve accurate segmentation of suspected ramp points, effectively filter out sparse noise points within the segments, ensure the spatial correlation of each point cloud segment, provide high-quality independent analysis units for subsequent steps such as geometric feature extraction and angle calculation, and avoid judgment errors caused by the mixing of point clouds in different regions.

[0321] In an optional embodiment, step 503 of the method provided in this application, which determines whether the current point cloud meets the segmentation condition based on the spatial continuity measure between adjacent point clouds, includes:

[0322] Step 561: If the Euclidean distance between the current point cloud data and the adjacent point cloud data exceeds the preset maximum distance threshold, then the segmentation condition is met.

[0323] For example, if the Euclidean distance between the current point cloud and the adjacent point cloud exceeds a preset maximum distance threshold, segmentation is triggered.

[0324] Please refer to the 3D point cloud data shown in Figure 8. The green box area in Figure 8 is used to visually demonstrate the specific scenario where "the Euclidean distance between adjacent point clouds exceeds the preset maximum distance threshold," triggering segmentation. This is applicable to scenarios encountered by mobile devices during travel, such as ramps with gaps, ramp surfaces with obvious depressions, or ramps connecting to ground debris (not obstacles, and not exceeding height limits). In this diagram, the point clouds are distributed according to the scanning sequence (column-by-column) of a non-scanning single-frame pulse imaging sensor, corresponding to a suspected ramp area. However, there is a significant gap between two adjacent point clouds (arranged sequentially according to the scanning sequence, belonging to adjacent scan columns or different positions within the same column), and the spatial Euclidean distance between them exceeds the preset maximum distance threshold, triggering the segmentation condition.

[0325] The specific judgment process is illustrated in the diagram: The preset maximum distance threshold is set based on sensor accuracy and the lawn scene. When the non-scanning single-frame pulse imaging sensor scans the target area, suspected slope points are input point by point and added to the point cloud queue. When processing two adjacent point clouds with a gap (point clouds with blank areas in the diagram), the Euclidean distance (i.e., the straight-line distance between the two points) in the sensor coordinate system is calculated. For example, if the calculated distance is 0.12m, exceeding the preset maximum distance threshold of 0.08m, it is determined that the two points have weak spatial correlation and do not belong to the same continuous slope or the same target, immediately triggering segmentation. After segmentation, the point clouds before and after the gap are divided into two independent effective segments, corresponding to the suspected slope areas on both sides of the gap, effectively avoiding feature extraction deviations caused by point cloud gaps. Simultaneously, isolated points at the gap (if they exist) can be included in the corresponding segment or judged as noise, further improving the rationality and accuracy of segmentation.

[0326] In an optional embodiment, step 553 of the method provided in this application, after determining whether the current point cloud meets the segmentation condition based on the spatial continuity metric between adjacent point clouds, includes:

[0327] Step 571: Obtain the mean of the historical direction vectors in the first point cloud sequence;

[0328] Step 572: Calculate the current direction vector formed between the current point cloud and its neighboring point clouds;

[0329] Step 573: Normalize the historical direction vector mean and the current direction vector respectively, and calculate the angle between them. If the angle exceeds the preset direction threshold, the segmentation condition is met.

[0330] The current direction vector formed by the current point cloud and its neighboring point clouds refers to the directed line segment formed by the current input point cloud and its neighboring previous point cloud (or several previous point clouds) in the scanning sequence. This vector represents the instantaneous extension direction of the point cloud sequence within a local range.

[0331] The normalization process described above, specifically for directional vector normalization, involves dividing a non-zero vector by its magnitude (length) to reduce its length to 1, resulting in a unit vector that retains only directional information and not magnitude. In other words, after normalization, all vectors are scaled to a length of 1, and only the direction of the arrows is compared. Regardless of the density of the point cloud, as long as the directions are the same, the angle is 0°, perfectly eliminating the influence of spacing differences. This normalization process enables accurate identification of key terrain boundaries such as wall corners and slope transitions, providing high-quality point cloud segmentation for subsequent slope analysis and traffic decisions.

[0332] For example, if the angle between the current direction vector and the average of the historical direction vectors exceeds a preset direction threshold (15°-30°), segmentation is triggered; if the first type of point cloud data (obstacle point cloud) is detected, segmentation is triggered.

[0333] For example: The first point cloud sequence includes a queue of direction vectors of length 5. Whenever a new point cloud P_i is input, its vector v_i relative to the previous point P_{i-1} is calculated and added to the queue, while the oldest vector is removed. The mean v_avg of the 5 vectors in the queue is calculated. L2 normalization is performed on v_i and v_avg to obtain unit vectors v_i_hat and v_avg_hat. The angle θ between them is calculated using the inverse cosine of their dot product. If θ > 30°, the point cloud extension direction is considered to have undergone a significant inflection point, this point is determined as the segment boundary, the current point cloud segment ends, and a new segment begins.

[0334] Please refer to the 3D point cloud data shown in Figure 9. The red box area in the schematic diagram of Figure 9 is used to visually illustrate the specific scenario where a sudden change in point cloud direction triggers segmentation. This is adapted to scenarios where a continuous slope encounters turning or bending during the movement of a mobile device, or where the slope connects with a low obstacle. In this schematic diagram, the point cloud is distributed sequentially according to the scanning sequence (column by column) of a non-scanning single-frame pulse imaging sensor, forming continuous point cloud segments, corresponding to a continuous suspected slope area. The first half of the point cloud segments shows a gentle extension trend, corresponding to the normal extension direction of a gentle slope. The second half of the point cloud segments shows obvious curvature and bending, and the extension direction is significantly deviated, that is, a sudden change in point cloud direction triggers the segmentation condition.

[0335] The specific judgment process is illustrated in the diagram: Assuming a non-scanning single-frame pulse imaging sensor scans a ramp, suspected ramp points acquired sequentially are stored in a point cloud queue and a point cloud direction queue. The historical average direction value is calculated using the direction vector formed by the first n adjacent points. When scanning the point cloud at the bend of the ramp (the area marked in the diagram), the current direction vector formed by that point and the previous point is calculated. After normalizing the current direction vector and the historical average direction vector, the angle between them is calculated. If this angle exceeds a preset direction threshold, it is determined that a sudden change in point cloud direction has occurred, immediately triggering segmentation. After segmentation, the point cloud before and after the ramp bend is divided into two independent valid segments: the first half is the point cloud segment of a gentle ramp, and the second half is the point cloud segment of the ramp after the turn or the connecting area. This avoids mixing of point clouds with different extension directions, ensuring the accuracy of subsequent angle calculations and ramp determination.

[0336] In an optional embodiment, step 523 of the method provided in this application, which identifies the target object in the target scene, includes:

[0337] Step 581: Convert the point cloud data of each segment from the radar coordinate system to the world coordinate system;

[0338] For example, the point cloud segment is transformed from the sensor coordinate system that moves with the mobile device to a fixed world coordinate system through a preset rigid body transformation matrix, so as to eliminate the interference of the device's own pose change on the point cloud geometry analysis.

[0339] Step 582: Calculate the spatial centroid of the point cloud data of each segment in the world coordinate system, and decentralize the point cloud coordinates.

[0340] By performing decentralization on all point cloud coordinates (i.e., subtracting the centroid coordinate from each point coordinate), the influence of the absolute position of the point cloud segments is eliminated, allowing subsequent analysis to focus on their relative spatial distribution.

[0341] Step 583: Construct a covariance matrix based on the decentralized point cloud data and perform eigenvalue decomposition, taking the eigenvector corresponding to the largest eigenvalue as the main spatial direction;

[0342] For example, based on the decentralized point cloud data, a 3×3 covariance matrix is ​​constructed, and eigenvalue decomposition is performed on this matrix. The unit eigenvector corresponding to the largest eigenvalue is extracted. This vector represents the distribution direction of the point cloud with the largest variance in space, which is defined as the spatial principal direction of the point cloud segment. The spatial principal direction of this point cloud segment usually corresponds to the main extension trend of the ramp surface or obstacle surface in physical terms.

[0343] Step 584: Calculate the angle between the main spatial direction and the vertical axis of the world coordinate system as the slope angle.

[0344] This included angle is the slope angle of the terrain represented by the point cloud segment. This angle directly and quantitatively reflects the degree of inclination of the target terrain.

[0345] The point cloud segments are transformed from the sensor coordinate system that moves with the mobile device to a fixed world coordinate system through a preset rigid body transformation matrix. This eliminates the interference of the device's own pose changes on the geometric shape analysis of the point cloud and ensures that subsequent feature extraction has global consistency and comparability.

[0346] For example: Steps 571-574 are performed on the suspected ramp point cloud obtained through segmentation in the above steps. The calculated angle between its principal spatial direction and the Z-axis of the world coordinate system is 60°. Therefore, the slope of this terrain segment is 90° - 60° = 30°. If the preset maximum passable slope angle for the device is 30°, then this terrain segment can be determined as a passable ramp; if the slope angle exceeds this threshold, it is determined as an impassable steep slope or obstacle.

[0347] The method provided in this application transforms the spatial features of point clouds into quantifiable slope angles, providing clear and verifiable criteria for accessibility decisions and improving the reliability and interpretability of the algorithm.

[0348] In optional embodiments, the method provided in this application may further include path planning after determining the type of point cloud data, such as:

[0349] Based on the spatial distribution information of the first type of point cloud data in the target area, an obstacle avoidance path for the automatic lawnmower is generated. The spatial distribution information includes the three-dimensional coordinates, distribution range, and relative distance of the first type of point cloud data to the automatic lawnmower. Combined with the automatic lawnmower's operation path planning rules (such as shortest path, operation along the boundary, obstacle avoidance priority), all impassable areas are avoided, and an optimal operation obstacle avoidance path that fits the operation requirements and is safe and feasible is generated.

[0350] Obviously, the first type of point cloud data can include the point cloud data initially screened in step 522, as well as the data determined in step 525.

[0351] For example, an obstacle avoidance path can be generated in real time for the self-moving device to execute, based on the obstacle space occupied by the first type of point cloud data and the physical size and safety margin of the self-moving device.

[0352] The above path planning methods can include:

[0353] The first type of point cloud data is projected onto a horizontal plane (XY plane) to form a two-dimensional obstacle distribution map. The occupied area of ​​each obstacle can be characterized by calculating its convex hull, circumscribed rectangle, or by rasterization. Areas in the currently scanned region that are not considered obstacles (i.e., not part of the first type of point cloud data) and are determined by slope are defined as passable areas. Combined with a preset or real-time constructed global operation boundary, a dynamic passable map for the current moment is formed. Based on the obstacle-occupied areas and passable areas, a real-time path planning algorithm (such as Vector Field Histogram (VFH), Dynamic Window Method (DWA), or its variants) is used to calculate a collision-free path for the self-moving device from its current position to the target position. The core constraint of path planning is to ensure that the outline of the self-moving device (including safety margin) is always within the passable area and far away from the obstacle-occupied areas.

[0354] The method provided in this application realizes a closed loop from environmental perception to motion control by directly and efficiently converting the accurate perception results (first type of point cloud data) of the front end into control commands (obstacle avoidance path) of the back end.

[0355] In an optional embodiment, referring to the logical diagram shown in Figure 10, the method provided in this application embodiment further includes the following processing steps:

[0356] Step S511: Obtain single-frame point cloud data of the target scene;

[0357] The point cloud data is generated based on the scanning time sequence of LiDAR.

[0358] Step S512: Perform noise filtering and redundant point cloud cleaning on the single frame point cloud data, including distance filtering and downsampling processing, to remove noise points, isolated points and redundant information that is irrelevant to the operation scene;

[0359] Step S513: Perform rasterization and ground height correction on the filtered point cloud. Project the point cloud onto the XY plane and divide it into 2D grids. Record the minimum Z value of each grid as the initial ground height. Correct the ground height based on the radar installation height and near-field obstruction to establish a unified ground height reference.

[0360] Step S514: Based on the corrected ground height reference and the preset obstacle height threshold, the point cloud is traversed point by point, and the difference between the Z value of each point cloud and the ground height of the corresponding grid is calculated. If the difference is higher than the obstacle height threshold, it is marked as a first type of point cloud (e.g., obstacle point); otherwise, it is marked as a second type of point cloud (e.g., suspected ramp point).

[0361] Step S515: Perform segmentation processing on the second type of point cloud. Based on the original scanning time sequence characteristics of the point cloud, divide the point cloud into multiple independent valid segments through Euclidean distance threshold, orientation change detection or multi-condition triggering mechanism to ensure that each segment of the point cloud has spatial continuity.

[0362] Step S516: Perform coordinate system transformation on each effective segment to transform the point cloud from the radar coordinate system to the world coordinate system in order to eliminate the interference of equipment movement on the spatial features of the point cloud;

[0363] Step S517: Based on the point cloud coordinates in the world coordinate system, extract the geometric features of each effective segment and determine the main extension direction of the point cloud segment;

[0364] Step S518: Calculate the tilt angle of the point cloud segment based on the angle between the main extension direction and the vertical direction (e.g., the Z-axis of the world coordinate system);

[0365] Step S519: Compare the tilt angle of the point cloud segment with the preset maximum tilt angle threshold for the self-moving device. If the tilt angle exceeds the threshold, the segment is determined to be an obstacle or a steep slope and output to the obstacle point cloud set. Otherwise, it is filtered out as a passable ramp.

[0366] The method provided in this application embodiment uses a two-layer obstacle judgment logic of "rapid initial screening + fine analysis" to distinguish between slope ground and real obstacles in real time and accurately at the front end of radar point cloud data stream, thereby improving the environmental perception accuracy, operation efficiency and system reliability of self-moving equipment in complex terrain.

[0367] Weather conditions include at least one of rainy or foggy weather. In rainy weather, on the one hand, the point cloud data collected by the sensors decreases, affecting the positioning function of the automated mobile device; on the other hand, the wheels of the automated mobile device may become muddy, causing slippage and affecting its movement. In foggy weather, the point cloud data collected by the lidar will generate a large amount of "fog point" noise, severely interfering with the environmental perception and decision-making of the automatic lawnmower, affecting the machine's normal operation. Therefore, automated mobile devices need to be equipped with rain and fog detection functions to prevent rain and fog from affecting the machine's positioning or movement, and thus affecting its normal operation.

[0368] Among related technologies, fog detection methods for self-moving devices have low detection accuracy, poor robustness, and low reliability, and are prone to false triggering or failure to trigger in some scenarios.

[0369] To address the aforementioned issues, in some embodiments, step S102 includes: identifying foggy weather based on three-dimensional point cloud data according to a sixth preset condition; wherein the sixth preset condition includes at least one of the following: the number of fog points in the three-dimensional point cloud data, the number of foggy weather zones, the proportion of foggy weather zones, the number of distant points, the number of ground points, and the number of invalid points; wherein, fog points are determined based on the distance between the point and the sensor origin and the point's reflection intensity; the horizontal field of view of the non-scanning single-frame pulse imaging sensor is divided into multiple zones, and each zone is determined to be a foggy weather zone based on the number of fog points in each zone; the proportion of foggy weather zones is the ratio of the number of foggy weather zones to the total number of zones; distant points are points whose distance from the sensor origin is greater than or equal to a distant distance threshold; ground points are points whose height is less than a ground height threshold; invalid points are points whose spatial coordinate components are all 0 and whose reflection intensity is also 0. Specifically, a fog point is a valid point whose distance from the sensor origin is less than or equal to the near distance threshold and whose reflection intensity is less than or equal to the low intensity threshold; wherein, a valid point is a point whose coordinate components in the spatial coordinates are at least non-zero, or whose reflection intensity is non-zero.

[0370] In some embodiments, referring to FIG11, a control method for a self-moving device includes:

[0371] S61. Acquire single-frame 3D point cloud data collected by a non-scanning single-frame pulse imaging sensor;

[0372] S62. Based on single-frame 3D point cloud data, determine at least one of the following: number of fog points, number of foggy zones, proportion of foggy zones, number of distant points, number of ground points, and number of invalid points;

[0373] S63. Based on at least one of the following: number of fog points, number of foggy zones, proportion of foggy zones, number of distant points, number of ground points, and number of invalid points, detect whether a single frame point cloud meets the foggy conditions.

[0374] Fog points are determined based on their distance from the sensor origin and their reflection intensity. The sensor's horizontal field of view is divided into multiple zones, and each zone is classified as a foggy zone based on the number of fog points within it. The proportion of foggy zones is the ratio of the number of foggy zones to the total number of zones. Far-distance points are defined as points whose distance from the sensor origin is greater than or equal to a far-distance threshold. Ground points are defined as points whose height is less than a ground height threshold. Invalid points are points whose spatial coordinate components are all 0 and whose reflection intensity is also 0. The sensor origin is the origin of the sensor coordinate system established based on a non-scanning single-frame pulse imaging sensor.

[0375] The inventors discovered that the number of fog points, the number of foggy zones, the percentage of foggy zones, the number of distant points, the number of ground points, and the number of invalid points can all characterize whether the current weather condition is foggy to a certain extent. It can be understood that this embodiment detects fog based on one or more of these factors, which improves detection accuracy, avoids false triggers or non-triggers, and enhances reliability. Furthermore, it improves robustness and is applicable to fog detection in various scenarios.

[0376] Furthermore, a fog point is defined as: a valid point whose distance from the sensor origin is less than or equal to a near-distance threshold and whose reflection intensity is less than or equal to a low-intensity threshold; wherein, a valid point is: a point whose coordinate components in the spatial coordinates are at least non-zero, or whose reflection intensity is non-zero.

[0377] The point cloud data collected by the sensor includes the spatial coordinates (x, y, z) and reflection intensity of the points. Based on the spatial coordinates (x, y, z), the distance between the point and the sensor origin, and the point's height can be calculated. Based on the distance between the point and the sensor origin and the point's reflection intensity, fog points can be identified and their number counted. Based on the number of fog points in each zone, it can be determined whether each zone is a foggy zone, and the number of foggy zones can be determined. Based on the number of foggy zones and the total number of zones, the percentage of foggy zones can be calculated. Based on the distance between the point and the sensor origin, distant points can be identified and their number counted. Based on the point's height, ground points can be identified and their number counted. Based on the spatial coordinates and reflection intensity of the points, invalid points can be identified and their number counted.

[0378] In some embodiments, whether a single-frame point cloud meets the foggy conditions is determined based on at least two of the following: the number of fog points, the number of foggy zones, the proportion of foggy zones, the number of distant points, the number of ground points, and the number of invalid points. As one example, the determination is made based on the number of fog points, the number of foggy zones, the number of distant points, the number of ground points, and the number of invalid points. As another example, the determination is made based on the number of fog points, the proportion of foggy zones, the number of distant points, the number of ground points, and the number of invalid points. As yet another example, the determination is made based on the number of foggy zones, the number of distant points, and the number of ground points. As yet still another example, the determination is made based on the number of foggy zones and the number of distant points.

[0379] Furthermore, step S63 includes: determining that a single-frame point cloud satisfies the foggy weather condition in response to satisfying at least one of the following conditions;

[0380] Foggy weather condition 1: The number of fog points is greater than or equal to the fog point count threshold;

[0381] Foggy weather condition 2: The number of foggy weather zones is greater than or equal to the zone number threshold;

[0382] Foggy day condition 3: The proportion of foggy areas is greater than or equal to the proportion threshold of the areas;

[0383] Foggy weather condition 4: The number of distant points is less than or equal to the threshold number of distant points;

[0384] Foggy weather condition 5: The number of ground points is less than or equal to the ground point number threshold;

[0385] Foggy weather condition 6: The number of invalid points is greater than or equal to the invalid point threshold.

[0386] The near-distance threshold is configured to be 0.5–1.5m; for example, it can be 0.5m, 0.6m, 0.7m, 0.8m, 0.9m, 1m, 1.1m, 1.2m, 1.3m, 1.4m, 1.5m, etc. The low-intensity threshold is configured to be 10–30; for example, it can be 10, 15, 20, 25, 30, etc. The far-distance threshold is configured to be 2–6m; for example, it can be 2m, 3m, 4m, 5m, 6m, etc. The height threshold is configured to be 0.5–2m; for example, it can be 0.5m, 0.75m, 1m, 1.25m, 1.5m, 1.75m, 2m, etc. The fog point count threshold is configured to 600–2000; for example, it can be 600, 750, 800, 1000, 1200, 1300, 1500, 1800, 2000, etc. The partition count threshold is configured to 3–8; for example, it can be 3, 4, 5, 6, 7, 8, etc. The partition percentage threshold is configured to 0.3–0.8; for example, it can be 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, etc. The long-distance point count threshold is configured to 1300–4000; for example, it can be 1300, 1500, 1800, 2000, 2200, 2500, 2800, 3000, 3200, 3500, 4000, etc. The invalid point threshold is configured to be between 6500 and 20000; for example, it can be 6500, 7500, 10000, 12000, 15000, 18000, 20000, etc. The ground point threshold is configured to be between 600 and 2000; for example, it can be 600, 750, 800, 1000, 1200, 1300, 1500, 1800, 2000, etc. The total number of points in a single frame of 3D point cloud data from the sensor can be 13500, 27000, etc. It can be understood that the threshold values ​​of various parameters need to be adjusted according to the different total number of points returned by a single frame of 3D point cloud data.

[0387] In one specific embodiment, referring to Figure 12, the self-moving device is an automatic lawnmower, and the non-scanning single-frame pulse imaging sensor is specifically a solid-state area array lidar with a horizontal field of view of 120°. To avoid omissions in point cloud statistics due to bias, a horizontal angle range of 150° directly in front of the self-moving device is selected as the statistical interval, and this interval is divided into 10 equal 15° partitions. The near-distance threshold is configured to 1m; the low-intensity threshold is configured to 25; the far-distance threshold is configured to 5m; the height threshold is configured to 1m; the fog point count threshold is configured to 1800; the partition count threshold is configured to 5; the far-distance point count threshold is configured to 2800; the invalid point count threshold is configured to 12000; and the ground point count threshold is configured to 1800. The control method for the self-moving device includes:

[0388] Step 1: Acquire single-frame 3D point cloud data from a non-scanning single-frame pulse imaging sensor;

[0389] Step 2: Traverse the single-frame 3D point cloud data and count the number of fog points, the number of foggy zones, the number of distant points, the number of ground points, and the number of invalid points;

[0390] Step 3: If the following conditions are met simultaneously: the number of fog points is greater than 1800, the number of foggy zones is greater than 5, the number of distant points is less than 2800, the number of ground points is less than 1800, and the number of invalid points is greater than 12000, then the single-frame point cloud is judged to meet the foggy conditions; otherwise, the single-frame point cloud is judged not to meet the foggy conditions.

[0391] In some embodiments, the control method for the self-moving device further includes: determining whether the current weather state is foggy or non-foggy based on whether one or more frames of point cloud data meet the foggy conditions; in response to detecting foggy weather, increasing at least one of the following thresholds: fog point count threshold, partition count threshold, partition ratio threshold, long-distance point count threshold, and invalid point count threshold, and / or decreasing the ground point count threshold; in response to detecting non-foggy weather, decreasing at least one of the following thresholds: fog point count threshold, partition count threshold, partition ratio threshold, long-distance point count threshold, and invalid point count threshold, and / or increasing the ground point count threshold.

[0392] It's understandable that detection results based on a single frame of point cloud data may contain errors in some scenarios. Determining whether it's foggy based on multiple frames of point cloud data, compared to relying on a single frame, can further improve fog detection accuracy and avoid false triggers or non-triggers that could affect the normal operation of the self-moving device. Furthermore, by adjusting various thresholds when fog is detected to increase the requirements for non-foggy detection, and then restoring these thresholds when non-foggy conditions are detected, frequent stopping and resuming of the self-moving device can be avoided.

[0393] In some embodiments, the control method for the self-moving device further includes: determining that it is foggy in response to a single frame point cloud meeting foggy conditions, and controlling the self-moving device to stop moving or return to the charging station. In this embodiment, the self-moving device is controlled to perform corresponding actions based on the detection results of the single frame 3D point cloud data. As an example, taking an automatic lawnmower as an example, when the automatic lawnmower is away from the charging station, in response to a single frame point cloud meeting foggy conditions, determining that it is foggy, and controlling the automatic lawnmower to return to the charging station.

[0394] In other embodiments, the control method for the self-moving device further includes: acquiring at least one new frame of point cloud data in response to a single frame of point cloud data meeting fog conditions; detecting whether fog conditions are met based on the new at least one frame of point cloud data; determining that it is foggy weather in response to the new at least one frame of point cloud data meeting fog conditions, and controlling the self-moving device to stop moving or return to the charging station. In this embodiment, the self-moving device controls the self-moving device to perform corresponding actions based on the detection results of multiple frames of point cloud data. As an example, taking an automatic lawnmower as an example, when the automatic lawnmower is in a repositioning or positioning state, in response to detecting that a single frame of point cloud data meets fog conditions, acquiring a new frame of point cloud data; in response to the new frame of point cloud data meeting fog conditions, determining that it is foggy weather, and controlling the automatic lawnmower to stop moving.

[0395] Furthermore, in response to a single frame of point cloud data meeting foggy conditions, at least one new frame of point cloud data is acquired after a preset time interval. It should be noted that the preset time interval in this embodiment differs from the sensor's sampling frequency; it is greater than the sensor's sampling interval. For example, under normal conditions, the sensor acquires one frame of point cloud data every 6 seconds. When a single frame of point cloud data is detected to meet foggy conditions, a new frame of point cloud data is acquired after a 12-second interval. As an example, taking an automatic lawnmower as an example, when the automatic lawnmower is in repositioning or positioning mode, in response to the detection that a single frame of point cloud data meets foggy conditions, a new frame of point cloud data is acquired after a preset time interval. Since the new frame of point cloud data meets foggy conditions, it is determined to be foggy, and the automatic lawnmower is controlled to stop moving.

[0396] It is understandable that by performing a re-inspection after detecting that the fog conditions are met, on the one hand, the fog detection accuracy can be greatly improved by using multiple frames of point cloud to determine fog conditions; on the other hand, after the initial detection of fog conditions, control of the self-moving device is not immediately executed, but the detection time interval benchmark is adjusted, and a second fog detection is performed after a preset interval. This can filter out instantaneous interference, detect fog conditions in the time dimension through new point cloud, and only when the fog condition is confirmed to persist is it determined to be foggy, and control of the self-moving device is executed. This can greatly improve the fog detection accuracy and avoid false triggering that affects the working efficiency of the self-moving device.

[0397] In some embodiments, referring to FIG13, the control method for the self-moving device further includes: when the self-moving device is in a state of leaving the charging pile, in response to detecting that a single frame of point cloud meets the fog conditions, controlling the self-moving device to return to the charging pile; when the self-moving device is in a relocation or positioning state, in response to detecting that a single frame of point cloud meets the fog conditions, acquiring at least one new frame of point cloud data, in response to the new at least one frame of point cloud data meeting the fog conditions, determining that it is foggy, and controlling the self-moving device to stop moving.

[0398] It is understandable that different control methods are used depending on the working status of the self-moving device. When the self-moving device leaves the charging pile, before the machine starts working, a single frame point cloud detects fog and immediately controls the self-moving device to return to the charging pile, avoiding the machine leaving the pile to work in foggy weather. In the positioning and repositioning operation state, foggy weather is determined by using multiple frames of point cloud, which can avoid misjudgment that causes the machine to stop frequently and affect the operation efficiency.

[0399] In related technologies, self-moving devices can use their own sensors to collect point cloud data of the surrounding environment, determine the rain value based on the collected point cloud data, and then determine whether it is raining based on the rain value. Therefore, some sensors provide a rain value as a reference for detecting rainfall. However, this method has poor robustness and low reliability, and in some scenarios, it is easy to trigger an increase in the rain value, thus causing false triggering. The inventors' research found that the main reason for false triggering is local occlusion caused by leaves or dirt adhering to the sensor surface.

[0400] To address the aforementioned issues, in some embodiments, step S102 includes: identifying rainy days based on three-dimensional point cloud data according to a seventh preset condition; the seventh preset condition includes at least one of the following: the number of near-distance low-intensity points in the three-dimensional point cloud data, the number of invalid points, the number of far-distance points, the proportion of near-distance low-intensity point clouds, and the variance of point cloud distribution; wherein, near-distance low-intensity points are: points whose distance from the sensor origin is less than a near-distance threshold and whose intensity is less than a low-intensity threshold; invalid points are: points whose spatial coordinate components are all 0 and whose reflection intensity is 0; far-distance... Points are defined as follows: points whose distance from the sensor origin is greater than the far-distance threshold; the proportion of low-intensity point clouds at close range is the ratio of the number of valid points whose distance from the sensor origin is less than the close-distance threshold and whose intensity is less than the low-intensity threshold to the total number of valid points whose distance from the sensor origin is less than the close-distance threshold; the variance of point cloud distribution is used to characterize the uniformity of the distribution of valid points in each partition when the horizontal field of view of the non-scanning single-frame pulse imaging sensor is divided into multiple partitions; valid points are defined as points whose coordinate components in the spatial coordinates are at least non-zero, or whose reflection intensity is non-zero.

[0401] For example, taking an automatic lawnmower as the self-moving device and a solid-state area array LiDAR as the non-scanning single-frame pulse imaging sensor, the horizontal field of view of the LiDAR is 120°. To avoid omissions in point cloud statistics due to bias, a horizontal angle range of 150° directly in front of the self-moving device is selected as the statistical interval, and this interval is divided into 10 equal 15° partitions. The method for identifying rainy days based on 3D point cloud data includes: step 701, determining the number of near-range low-intensity points, the proportion of near-range low-intensity points, and the variance of point cloud distribution based on 3D point cloud data; step 702, if the variance of point cloud distribution is less than a set threshold (e.g., 15000), and the proportion of near-range low-intensity points is greater than a set threshold (e.g., 70%), and the number of near-range low-intensity points is greater than a set threshold (e.g., 100), then it is determined to be a rainy day. Of course, in other embodiments, the invalid point quantity condition and the far-range point quantity condition can also be combined or replaced to identify rainy days, which will not be elaborated here.

[0402] The sensor operating environment includes the sensor's occlusion status. Sensor occlusion affects the machine's perception of its environment, impacting positioning and path planning, and consequently, disrupting normal machine operation. Therefore, self-moving devices need to be equipped with sensor occlusion detection functionality to prevent sensor occlusion from affecting machine operation. However, current occlusion detection methods for self-moving devices suffer from low accuracy, poor robustness, and low reliability, and are prone to false triggering or failure to trigger in some scenarios.

[0403] To address the aforementioned issues, in some embodiments, step S102 includes: based on 3D point cloud data, identifying whether the non-scanning single-frame pulse imaging sensor is in an occluded state according to an eighth preset condition; the eighth preset condition includes at least one of: invalid point quantity condition, invalid point percentage condition, occluded point quantity condition, occluded point percentage condition, and non-occluded point quantity condition; wherein, an invalid point is a point whose spatial coordinate components are all 0 and whose reflection intensity is also 0; the invalid point percentage is the ratio of the number of invalid points to the total number of points in the single-frame 3D point cloud data; occluded points and non-occluded points are determined based on the distance between the point and the sensor origin and / or the spatial coordinates of the point, wherein the distance between the occluded point and the sensor origin is less than the distance between the non-occluded point and the sensor origin; the occluded point percentage is the ratio of the number of occluded points to the number of valid points in the single-frame 3D point cloud data; a valid point is a point whose spatial coordinate components are at least non-zero or whose reflection intensity is non-zero. Specifically, occluded points include points whose distance from the sensor origin meets the short distance condition; unoccluded points are points whose distance from the sensor origin is greater than or equal to the long distance threshold.

[0404] In some embodiments, referring to FIG14, a control method for a self-moving device includes:

[0405] S81. Acquire single-frame 3D point cloud data collected by a non-scanning single-frame pulse imaging sensor;

[0406] S82. Based on single-frame 3D point cloud data, determine at least one of the following: number of invalid points, percentage of invalid points, number of occluded points, percentage of occluded points, and number of non-occluded points;

[0407] S83. Detect whether a single frame point cloud meets the occlusion condition based on at least one of the following: number of invalid points, percentage of invalid points, number of occluded points, percentage of occluded points, and number of non-occluded points.

[0408] Invalid points are defined as points where all spatial coordinate components are 0 and the reflection intensity is also 0. The invalid point ratio is the ratio of the number of invalid points to the total number of points in a single frame of 3D point cloud data. Occluded and non-occluded points are determined based on the distance between the point and the sensor origin and / or the point's spatial coordinates; the distance between an occluded point and the sensor origin is less than the distance between a non-occluded point and the sensor origin. The occluded point ratio is the ratio of the number of occluded points to the number of valid points in a single frame of 3D point cloud data. Valid points are defined as points where at least one spatial coordinate component is not 0, or where the reflection intensity is not 0. The sensor origin is the origin of the sensor coordinate system established based on a non-scanning single-frame pulse imaging sensor.

[0409] The inventors discovered that the number of invalid points, the percentage of invalid points, the number of occluded points, the percentage of occluded points, and the number of unoccluded points can all characterize the occlusion of a non-scanning single-frame pulse imaging sensor to a certain extent. It can be understood that this embodiment detects the occlusion state of the non-scanning single-frame pulse imaging sensor based on one or more of these metrics, which can improve detection accuracy, avoid false triggering or non-triggering, and improve reliability. Furthermore, it can improve robustness and is applicable to occlusion detection in various scenarios.

[0410] Further, the occlusion point includes points whose distance from the sensor origin meets a proximity condition. As an example, the proximity condition is configured such that the distance between the point and the sensor origin is less than or equal to a proximity threshold. Specifically, the proximity threshold is configured to be 0.03–0.1 m. Exemplarily, the proximity threshold could be 0.03 m, 0.04 m, 0.05 m, 0.06 m, 0.07 m, 0.08 m, 0.09 m, 0.1 m, etc. As another example, the distance between the point and the sensor origin falls within a preset distance range. Specifically, the preset distance range is configured to fall within the range of 0–0.1 m. Exemplarily, the preset distance range could be 0–0.03 m, 0–0.04 m, 0–0.05 m, 0–0.06 m, 0–0.07 m, 0–0.08 m, 0–0.09 m, 0–0.1 m, etc.

[0411] In some embodiments, the occlusion point further includes a point located on the side opposite to the sensing direction of the lidar sensor from the sensor origin. That is, a point whose x-axis coordinate is less than 0. It can be understood that points at close range and points behind the lidar sensor can both characterize the laser beam being blocked to some extent, and therefore are used as occlusion points.

[0412] Furthermore, an unobstructed point is defined as a point whose distance from the sensor origin is greater than or equal to a long-distance threshold. Specifically, the long-distance threshold is configured to be between 0.15 and 0.5 m. For example, the long-distance threshold can be 0.15 m, 0.2 m, 0.25 m, 0.3 m, 0.35 m, 0.4 m, 0.45 m, 0.5 m, etc.

[0413] Point cloud data acquired by a non-scanning single-frame pulse imaging sensor includes the spatial coordinates (x, y, z) and reflection intensity of each point. The distance between the point and the sensor origin can be calculated based on the spatial coordinates (x, y, z). Invalid and valid points can be identified based on the spatial coordinates (x, y, z) and reflection intensity, and the number of invalid and valid points can be counted. Occluded and unoccluded points can be identified based on the distance between the point and the sensor origin and the point's spatial coordinates, and the number of occluded and unoccluded points can be counted. The proportion of occluded points can be calculated based on the number of occluded and valid points. The proportion of invalid points can be calculated based on the number of invalid points and the total number of points in a single frame of 3D point cloud data. The total number of points in a single frame of 3D point cloud data is a fixed value, but may vary depending on the specific non-scanning single-frame pulse imaging sensor used.

[0414] In some embodiments, whether a single-frame point cloud meets the occlusion condition is determined based on at least two of the following: the number of invalid points, the percentage of invalid points, the number of occluded points, the percentage of occluded points, and the number of non-occluded points. As one example, the occlusion condition is determined based on the number of invalid points, the number of occluded points, and the number of non-occluded points. As another example, the occlusion condition is determined based on the percentage of invalid points, the percentage of occluded points, and the number of non-occluded points. As yet another example, the occlusion condition is determined based on the number of invalid points, the percentage of invalid points, the number of occluded points, the percentage of occluded points, and the number of non-occluded points. As yet still another example, the occlusion condition is determined based on the number of occluded points and the number of non-occluded points.

[0415] Further, step S83 includes: determining that a single frame point cloud satisfies an occlusion condition in response to satisfying at least one of the following conditions;

[0416] Occlusion condition 1: The number of invalid points is greater than or equal to the invalid point count threshold;

[0417] Occlusion condition 2: The proportion of invalid points is greater than or equal to the invalid point proportion threshold;

[0418] Occlusion condition 3: The number of occlusion points is greater than or equal to the threshold number of occlusion points;

[0419] Occlusion condition 4: The proportion of occluded points is greater than or equal to the threshold for the proportion of occluded points;

[0420] Occlusion condition 5: The number of unoccluded points is less than or equal to the threshold number of unoccluded points.

[0421] The invalid point threshold is configured to be between 5000 and 20000; for example, it can be 5000, 6000, 7000, 8000, 10000, 12000, 16000, 18000, 20000, etc. The invalid point percentage threshold is configured to be between 40% and 80%; for example, it can be 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, etc. The occlusion point threshold is configured to be between 500 and 2000; for example, it can be 500, 600, 700, 800, 1000, 1200, 1600, 1800, 2000, etc. The threshold for the percentage of occluded points is configured to be 4%–8%; for example, it could be 4%, 4.5%, 5%, 5.5%, 6%, 6.5%, 7%, 7.5%, 8%, etc. The threshold for the number of non-occluded points is configured to be 100–400; for example, it could be 100, 200, 300, 400, etc. The total number of points in a single frame of 3D point cloud data could be 13,500, 27,000, etc. It is understood that the threshold values ​​of each parameter need to be adjusted depending on the total number of points returned from a single frame of 3D point cloud data.

[0422] The non-scanning single-frame pulse imaging sensor is a solid-state area array lidar. The horizontal field of view of a solid-state area array lidar is 75°–150°, and the vertical field of view is 15°–90°. For example, the horizontal field of view of a solid-state area array lidar can be 75°, 90°, 120°, 135°, 150°, etc., and the vertical field of view can be 15°, 30°, 45°, 50°, 75°, 90°, etc. As a specific example, the horizontal field of view of a solid-state area array lidar is 120°, and the vertical field of view is 50°. As another specific example, the horizontal field of view of a solid-state area array lidar is 150°, and the vertical field of view is 75°.

[0423] In one specific embodiment, referring to FIG15, the self-moving device is an automatic lawnmower. Based on the number of invalid points, the number of occluded points, and the number of unoccluded points, it detects whether a single frame of the point cloud meets the occlusion condition. The invalid point threshold is configured to 10000, the occluded point threshold is configured to 800, and the unoccluded point threshold is configured to 200. The control method for the self-moving device includes:

[0424] Step 1: Acquire single-frame 3D point cloud data from a non-scanning single-frame pulse imaging sensor;

[0425] Step 2: Traverse the single frame of 3D point cloud data and count the number of invalid points, occluded points, and non-occluded points;

[0426] Step 3: If the number of invalid points is greater than 1000, the number of occluded points is greater than 800, and the number of non-occluded points is less than 200, then the single frame point cloud is judged to meet the occlusion conditions; otherwise, the single frame point cloud is judged not to meet the occlusion conditions.

[0427] In some embodiments, the control method for the self-moving device further includes: detecting whether the current state of the non-scanning single-frame pulse imaging sensor is occluded or unoccluded based on whether one or more frames of point cloud data meet the occlusion condition; in response to detecting occlusion, increasing at least one of the invalid point count threshold, invalid point percentage threshold, occluded point count threshold, and occluded point percentage threshold, and / or decreasing the unoccluded point count threshold; in response to detecting unocclusion, decreasing at least one of the invalid point count threshold, invalid point percentage threshold, occluded point count threshold, and occluded point percentage threshold, and / or increasing the unoccluded point count threshold. For example, occlusion is detected based on the number of invalid points, the number of occluded points, and the number of unoccluded points. In the unoccluded state, the invalid point count threshold is 1000, the occluded point count threshold is 1000, and the unoccluded point count threshold is 200; in the occluded state, the invalid point count threshold is 8000, the occluded point count threshold is 800, and the unoccluded point count threshold is 200. That is, in this embodiment, only the threshold for invalid points and the threshold for occluded points are adjusted, while the threshold for non-occluded points is not adjusted.

[0428] It is understandable that in some scenarios, detection results based on a single frame of point cloud data may contain errors. Determining whether an occlusion occurs based on multiple frames of point cloud data can further improve occlusion detection accuracy compared to relying on a single frame, avoiding false triggers or non-triggers that could affect the normal operation of the self-moving device. Furthermore, by adjusting various occlusion thresholds when occlusion is detected to increase the requirements for non-occlusion detection, and then restoring these thresholds when non-occlusion is detected, frequent stopping and resuming of the self-moving device can be avoided.

[0429] In some embodiments, referring to FIG16, the control method for the self-moving device further includes: in response to a single frame of point cloud satisfying an occlusion condition, acquiring at least one new frame of point cloud data; detecting whether an occlusion condition is satisfied based on the new at least one frame of point cloud data; and in response to the new at least one frame of point cloud data satisfying an occlusion condition, determining that occlusion has occurred, and controlling the self-moving device to stop moving. In this embodiment, the self-moving device controls the self-moving device to perform corresponding actions based on the detection results of multiple frames of point cloud data.

[0430] Furthermore, in response to a single frame of point cloud meeting the occlusion condition, at least one new frame of point cloud data is acquired after a preset time interval. It should be noted that the preset time interval in this embodiment differs from the sampling frequency of the non-scanning single-frame pulse imaging sensor itself; it is greater than the sampling interval of the non-scanning single-frame pulse imaging sensor itself. For example, the sensor acquires one frame of point cloud data every 3 seconds under normal conditions. When it detects that a single frame of point cloud meets the occlusion condition, a new frame of point cloud data is acquired after a 6-second interval. As an example, taking an automatic lawnmower as an example of a self-moving device, in response to the detection that a single frame of point cloud meets the occlusion condition, a new frame of point cloud data is acquired after a preset time interval, such as 6 seconds. If the new frame of point cloud data meets the occlusion condition, it is determined to be occlusion, and the automatic lawnmower is controlled to stop moving.

[0431] It is understandable that by performing a re-inspection after detecting that the occlusion conditions are met, on the one hand, the accuracy of occlusion detection can be greatly improved by using multiple frames of point cloud to determine occlusion; on the other hand, after the initial detection of occlusion conditions, control of the self-moving device is not immediately executed, but the detection time interval benchmark is adjusted, and a second occlusion detection is performed after a preset interval. This can filter out instantaneous interference, detect occlusion through new point cloud in the time dimension, and only when occlusion is detected again is it determined to be occlusion and control of the self-moving device is executed. This can greatly improve the accuracy of occlusion detection and avoid false triggering that affects the working efficiency of the self-moving device.

[0432] In other embodiments, the control method for the self-moving device further includes: determining occlusion in response to a single frame of point cloud data meeting the occlusion condition, and controlling the self-moving device to stop moving. In this embodiment, the self-moving device executes corresponding actions based on the detection results of the single frame of 3D point cloud data.

[0433] In some embodiments, referring to Figure 17, models can be established based on the point cloud distribution of occlusion, rain, and fog, according to the characteristics of the lidar, i.e., occlusion models, rain models, fog models, etc. During actual operation, feature analysis is performed on the 3D point cloud data, and the analysis results are compared with the occlusion model, rain model, and fog model respectively. When it matches the occlusion model, occlusion information is output; when it matches the rain model, rain information is output; and when it matches the fog model, fog information is output. The occlusion model includes partial occlusion models and full occlusion models to accurately identify occlusion conditions. This embodiment uses occlusion, rain, and fog as examples; similar methods can be used for other weather conditions, such as hail. Various extended functions can be realized through a non-scanning single-frame pulse imaging sensor.

[0434] To address the aforementioned problems, this application also provides a control method for a self-moving device. Referring to FIG18, the self-moving device includes: a non-scanning single-frame pulse imaging sensor configured to acquire information about the surrounding environment of the self-moving device and obtain three-dimensional point cloud data; and a state detection unit configured to detect motion state data of the self-moving device; the control method for the self-moving device includes:

[0435] S201. Acquire 3D point cloud data and motion state data;

[0436] S202. Align the 3D point cloud data and motion state data according to the timestamp to obtain the 3D point cloud data and motion state data at the same moment.

[0437] S203. Project the 3D point cloud data and motion state data at the same time onto the same coordinate system to obtain multi-source data at the same time and in the same coordinate system;

[0438] S204. Based on multi-source data, calculate and obtain the pose information of the self-moving device, and / or construct an environmental point cloud map.

[0439] The state detection unit includes an inertial measurement unit; motion state data includes acceleration and angular velocity. In some embodiments, the state detection unit further includes a wheeled odometer; motion state data also includes displacement and orientation angle.

[0440] Step S204 is further configured as follows: based on multi-source data, state estimation is performed through a fusion algorithm, and the pose information of the mobile device and / or the environmental point cloud map is output.

[0441] For example, as shown in FIG19, the control method for the self-moving device includes steps S21 to S26.

[0442] Step S21: During operation, the self-moving device acquires in real time the three-dimensional point cloud data of the surrounding environment collected by the lidar, the motion state data collected by the inertial measurement unit (i.e., inertial motion data), and the motion state data collected by the wheeled odometer (i.e., wheeled motion data).

[0443] Specifically, regarding 3D point cloud data: In this embodiment, a 3D solid-state area array lidar based on Flash technology acquires 3D point cloud data, and its working principle is similar to that of a 3D ToF depth camera. The lidar emits modulated light pulses towards the target area through a transmitting array, and the area array receiver (usually a CMOS or CCD sensor) receives the light signals reflected from the target. The depth value corresponding to each pixel is calculated based on the time-of-flight of the light pulse. Unlike traditional mechanical lidar that scans point by point, the area array lidar acquires all depth pixels within the entire field of view at once, forming a complete depth image (i.e., a point cloud). In this embodiment, the area array lidar has a field of view of 120° (horizontal) × 50° (vertical), a frame rate of 15Hz, and a ranging range of 0.2 meters to 8 meters. Each point cloud point contains three-dimensional spatial coordinates (x, y, z), and some lidars also output light intensity information and measurement confidence information. The point cloud data resides in the lidar's own sensor coordinate system. For example, the field of view of a solid-state area array lidar covers the direction of travel of the mobile device, enabling it to detect obstacles and ground information within a range of 0.2 meters to 8 meters in front. To obtain a wider coverage area, multiple solid-state area array lidars can also be installed around the mobile device.

[0444] Inertial Motion Data: Inertial motion data is acquired through an Inertial Measurement Unit (IMU), which typically integrates a three-axis micro-electro-mechanical systems (MEMS) accelerometer and a three-axis MEMS gyroscope. The accelerometer measures the acceleration of the self-moving device in three directions (forward-backward, left-right, and up-down), including the gravitational acceleration component; the gyroscope measures the angular velocity of the self-moving device about the three axes. The sampling frequency of the IMU is much higher than that of the lidar; in this embodiment, it is set to 200Hz. The raw data output by the IMU includes sensor bias error and measurement noise, which need to be compensated for in subsequent processing. The IMU data is located in the IMU's own sensor coordinate system.

[0445] Wheel motion data: Wheel motion data is collected via a wheel odometer, specifically implemented by incremental photoelectric encoders installed on the left and right drive wheels of the self-moving device. The encoders calculate the angle of rotation of the wheels by detecting the pulse signals generated when the wheels rotate. Combined with the known wheel diameter and gear ratio, the distance traveled by the self-moving device can be calculated. The steering angle of the self-moving device can also be calculated using the speed difference between the left and right wheels. The wheel odometer outputs the linear velocity and yaw rate of the self-moving device in its own coordinate system. The sampling frequency of the wheel odometer is typically around 50Hz. Note that the wheel odometer cannot measure vertical (up-down) motion, nor can it measure changes in roll and pitch angles.

[0446] Step S22: Align all acquired sensor data.

[0447] Because the three sensors have different sampling frequencies and each has its own independent time reference, directly using unaligned data for fusion will introduce time inconsistency errors. For example, associating IMU data from different times with point cloud data will lead to motion estimation errors. This step aligns all acquired sensor data, including timestamp alignment and coordinate system alignment.

[0448] Timestamp Alignment: First, a unified timestamp is applied to each sensor data packet. In this embodiment, the IMU timestamp is used as the reference because the IMU has the highest sampling frequency and can provide the finest temporal resolution. For point cloud data, each frame of data from the area array LiDAR has a start exposure timestamp and an end timestamp (the end of exposure is the start of data readout). In this embodiment, the midpoint timestamp is used as the timestamp for that frame of point cloud data, which is the average of the start and end times. For odometry data, each odometry data packet has its own timestamp. Since the time of the point cloud frame is usually not aligned with the IMU data time (point cloud frame rate 15Hz, IMU 200Hz, the point cloud frame time falls between two IMU data times), an interpolation method is required for synchronization. Specifically, the two times closest to the point cloud frame time are found in the IMU data sequence, and the IMU data (acceleration and angular velocity) at that time is calculated using linear interpolation. A similar linear interpolation method is used to synchronize the odometry data to the point cloud frame time. After timestamp alignment, each frame of point cloud data corresponds to a set of time-synchronized IMU data and odometry data.

[0449] Coordinate System Alignment: Data from different sensors reside in different coordinate systems. The area-array lidar is located in the radar coordinate system, with its origin at the radar's optical center, the X-axis pointing forward, the Y-axis pointing to the left, and the Z-axis pointing upward. The IMU is located in the IMU coordinate system, whose axis orientation may differ from the radar coordinate system. Wheeled odometer data is located in the self-moving device's coordinate system, with its origin typically located at the projection of the self-moving device's center onto the ground, the X-axis pointing in the forward direction, the Y-axis pointing to the left, and the Z-axis pointing vertically upward. To enable processing of all data within the same spatial framework, all data needs to be unified into the self-moving device's own coordinate system.

[0450] Therefore, the following external parameters were obtained in advance through offline calibration:

[0451] The transformation relationship from the radar coordinate system to the self-moving device coordinate system includes a 3×3 rotation matrix (describing the rotation relationship between the coordinate axes) and a 3×1 translation vector (describing the position offset between the origins).

[0452] The transformation relationship from the IMU coordinate system to the self-moving device coordinate system also includes the rotation matrix and translation vector.

[0453] Calibration is typically performed before the self-moving device leaves the factory or after the sensors are installed, and can be accomplished using existing open-source calibration tools (such as lidar_imu_calib, kalibr, etc.). During coordinate system transformation, for any point in the point cloud, multiplying its coordinates by a rotation matrix and adding a translation vector yields the point's coordinates in the self-moving device's coordinate system. For acceleration and angular velocity measured by the IMU, since they are vectors, only a rotation matrix is ​​needed for direction transformation; no translation transformation is required. Wheel odometry data is already in the self-moving device's coordinate system and requires no additional transformation. After coordinate system alignment, all sensor data are in a unified self-moving device coordinate system, laying the foundation for subsequent data fusion.

[0454] Step S23: Use inertial data to correct motion distortion of environmental point cloud data, and use inertial data to recursively calculate the predicted state variables of the self-moving device.

[0455] This step involves two closely related operations: using inertial data to correct motion distortion in the point cloud, and using inertial data to recursively predict the state variables of the self-moving device.

[0456] Motion distortion correction: Acquiring a frame of point cloud data from a LiDAR array requires a certain amount of time, typically tens of milliseconds, such as 50 milliseconds. During this time, the mobile device may be in motion, such as moving forward, turning, or bumping, causing each point in the point cloud data to actually correspond to the pose of the mobile device at different times. For example, the earlier exposed portion of a point cloud frame corresponds to the pose of the mobile device at time t0, while the later exposed portion corresponds to the pose of the mobile device at time t1. If this point cloud frame is directly matched with the map (assuming the entire point cloud frame corresponds to the same time), motion distortion errors will occur. The core of motion distortion correction is to use high-frequency IMU data to obtain the continuous pose changes of the mobile device during the scanning period, and transform each point in the point cloud to the coordinate system of the mobile device at the same reference time (usually the start time of the frame).

[0457] The specific correction process includes: First, obtaining the start time t of the point cloud frame. start and the end time t end The reference time is determined to be t. start Then, by integrating the angular velocity measured by the IMU, the value from t is obtained. start The rotational change up to the current time t; by double integration using the acceleration measured by the IMU (after subtracting the gravitational component), the value from t is obtained. startThe position change up to the current time t. Since the angular velocity and acceleration measurements of the IMU have zero bias, the current zero bias estimate needs to be subtracted before integration. For any point in the point cloud, its acquisition time is t. i t is usually taken start With t end For a Flash radar with a global shutter, where the exposure time of each pixel is almost the same, it can be assumed that all points have the same acquisition time. This value is typically taken as t. start and t end The midpoint. The pose change obtained through the above integration is used to transform this point to t. start In the coordinate system at time t, all points in the transformed point cloud correspond to t. start The pose of the self-moving device at any given time is used to eliminate intra-frame motion distortion. In practical engineering implementations, since the time span of the point cloud frame is relatively short (about 50 milliseconds), an approximation method can be used to reduce the computational load: assume that the self-moving device moves at a constant speed during this time period, use the average speed instead of the integral, or use only the attitude change of the IMU for rotation correction while ignoring translation correction, because translation distortion is relatively small.

[0458] Then, the predicted state variables of the self-moving device are recursively calculated using IMU data. The state variables of the self-moving device include: position (three-dimensional coordinates in the world coordinate system), velocity (three-dimensional velocity in the world coordinate system), attitude (represented by quaternions or rotation matrices), gyroscope bias, and accelerometer bias. The IMU measurement model is as follows: angular velocity measurement = true angular velocity + gyroscope bias + white noise; acceleration measurement = true acceleration + accelerometer bias + white noise - gravitational acceleration (in the self-moving device coordinate system).

[0459] The state prediction process involves: given the state variables from the previous moment, calculating the changes in the state variables based on the angular velocity and acceleration (excluding bias) measured by the IMU at the current moment, and then recursively deducing the state variables at the current moment. Specifically, the attitude change is obtained by integrating the angular velocity, the velocity change by integrating the acceleration, and the position change by integrating the velocity. This recursion does not require external observation and relies entirely on the IMU itself, thus it can be performed at very high frequencies, such as 200Hz. However, due to noise and bias drift in IMU measurements, the state variables derived purely from the IMU will gradually diverge over time, leading to error accumulation. Therefore, the predicted state variables require subsequent wheel odometry and point cloud observations for correction.

[0460] Step S24: Fuse the wheel motion data with the predicted state variables, and use the wheel motion observations to correct the predicted state variables to obtain the fused state estimate.

[0461] This step fuses the aligned wheel motion data with the predicted state variables obtained in step S23, and corrects the predicted state variables using the wheel motion observations to obtain the fused state estimate. The wheel odometer provides the linear velocity and yaw rate in the self-moving device's own coordinate system. This information is complementary to the state variables derived by the IMU: the wheel odometer's velocity measurement is relatively accurate with small errors in the short term, but it has cumulative errors and cannot measure the vertical direction; the IMU can measure complete six-axis motion, but it is subject to drift.

[0462] This embodiment employs the Error State Kalman Filter (ESKF) framework for fusion. The core of ESKF is to decompose the real state into a nominal state and an error state: the nominal state is directly derived from IMU measurements (ignoring noise), while the error state is estimated and corrected using Kalman filtering. The advantage of this decomposition is that the error state is typically very small and can be approximated using a linearized model, avoiding the complex calculations of nonlinear filtering; simultaneously, the attitude error can be represented by a three-dimensional vector instead of a quaternion, avoiding quaternion normalization constraints.

[0463] The specific fusion process includes: First, obtaining the covariance matrix of the nominal state variables (i.e., the IMU recursive results without considering noise) and the error state variables from step S23. Then, constructing the observation model for the wheeled odometer: the linear velocity measured by the wheeled odometer (in the self-moving device coordinate system) equals the true velocity plus measurement noise; the yaw rate measured by the wheeled odometer equals the true yaw rate plus measurement noise. Projecting the nominal state variables into the observation space yields the predicted observation values ​​(i.e., the odometer readings estimated based on the current state). Subtracting the predicted observation values ​​from the actual observation values ​​(the linear velocity and yaw rate of the self-moving device in the body coordinate system calculated by the wheeled odometer) yields the observation residuals. A linear relationship exists between the observation residuals and the error state variables, which can be described by the observation matrix.

[0464] Next, the Kalman gain is calculated. The essence of the Kalman gain is to balance the uncertainty of the prior state with the uncertainty of the observations. The larger the prior covariance (the more uncertain the state), the larger the Kalman gain, and the greater the weight of the observations in correcting the state; conversely, the greater the observation noise, the smaller the Kalman gain, and the smaller the weight of the observations in correcting the state. The correction amount for the error state variable is obtained by multiplying the Kalman gain by the observation residuals. This correction is added to the current error state variable, and the covariance matrix is ​​updated simultaneously. Finally, the updated error state variable is injected into the nominal state variable (i.e., the nominal state variable is corrected using the error state variable), and the error state variable is then set to zero.

[0465] It is important to note that wheeled odometers lack vertical position and velocity constraints, and also cannot provide roll and pitch angle information. Therefore, the state estimate after this fusion step is effectively constrained in the horizontal position and yaw angle directions, but still has significant uncertainties in the vertical, roll, and pitch directions. These uncertainties will be eliminated by subsequent point cloud observations.

[0466] Step S25: The fused state estimate is corrected using point cloud data through an iterative process, and the final pose of the self-moving device is output.

[0467] This step iteratively refines the fused state estimate obtained in step S24 using point cloud data. Point cloud data provides three-dimensional structural information of the environment; it is an absolute observation that does not accumulate drift over time, effectively correcting accumulated errors from the IMU and odometry. The core of this step is to project the point cloud onto the world coordinate system using the current state estimate, perform geometric matching with the constructed 3D map, and optimize the state quantity by minimizing the distance between the point cloud and the map.

[0468] Figure 20 shows a flowchart of the iterative correction process. As shown in Figure 20, the process includes the following sub-steps:

[0469] Sub-step S251: Point cloud feature extraction.

[0470] First, the motion distortion corrected point cloud is transformed from the self-moving device coordinate system to the world coordinate system using the current state estimate (nominal state value plus error state value). Since the point cloud of the area array lidar has a regular arrangement structure similar to an image, that is, each point has a row index and a column index, the curvature of each point can be estimated by using the relative positional relationship between each point and its neighboring points, thereby distinguishing different geometric features.

[0471] Curvature estimation specifically includes: for each point in the point cloud, selecting several neighboring points (e.g., 5 points each) on the left and right sides of the same row in the point cloud image. Calculating the composite direction vector of these neighboring points relative to the current point, or calculating the distance from the current point to the local plane fitted by the neighboring points. Points with larger curvature are located in regions with abrupt geometric changes, such as edges and corners; these points are called edge feature points. Points with smaller curvature are located in relatively flat planar regions; these points are called planar feature points. In this embodiment, to balance accuracy and efficiency, feature classification is not performed on all points; instead, a fixed number of feature points are extracted, for example, 100 edge feature points and 200 planar feature points per frame. During extraction, points are sorted according to curvature magnitude, selecting the points with the largest curvature as edge feature points and the points with the smallest curvature as planar feature points. Simultaneously, to avoid excessive concentration of feature points, non-maximum suppression can be used to ensure that feature points are spatially uniformly distributed.

[0472] Sub-step S252: Map matching and observation residual calculation.

[0473] For each extracted feature point, taking a planar feature point as an example, its coordinates in the world coordinate system are known. From the constructed 3D probabilistic voxel map, map points that are less than a first preset threshold (e.g., 0.5 meters) and have the same feature type (i.e., both are planar feature points) are searched. In addition to storing location coordinates, each map point in the 3D probabilistic voxel map also stores its confidence level (a measure of the reliability of cumulative observations) and feature type.

[0474] If the number of candidate map points found reaches the preset minimum number of matching points (e.g., 5), the feature point is considered suitable for matching. Then, the candidate map points are sorted from highest to lowest confidence, and the top few map points with the highest confidence (e.g., 5) are selected. For planar feature points, these high-confidence map points should approximately lie on the same plane. A plane is fitted using these points, including: first, calculating the centroid of all points; then, calculating the covariance matrix of each point relative to the centroid; performing eigenvalue decomposition on the covariance matrix; the eigenvector corresponding to the smallest eigenvalue is the normal vector of the plane, and the equation of the plane can be determined by the normal vector and the centroid. For the current feature point, its perpendicular distance to the fitted plane is calculated. If the current feature point is exactly on the plane, the distance is 0; otherwise, the distance is greater than 0. This distance is the observation residual, which reflects the deviation between the point cloud position predicted by the current state estimator and the actual map position.

[0475] Similarly, for edge feature points, find edge feature points with high confidence from the map, fit a straight line (take the eigenvector corresponding to the largest eigenvalue through principal component analysis as the direction vector, and calculate a point on the line), and calculate the distance from the current point to the fitted line as the observation residual.

[0476] There is a functional relationship between the observation residuals and the error state variables. Since the coordinates of feature points are obtained by coordinate transformation from the pose of the self-moving device and the original coordinates of the point cloud, the partial derivatives of the observation residuals with respect to the error state variables can be calculated using the chain rule. This partial derivative matrix (observation matrix) links the observation residuals to the error state variables that need to be estimated.

[0477] Sub-step S253: Objective function construction and Kalman update.

[0478] Based on the consistency constraint between prior state information and observation information, an objective function is constructed. This objective function consists of two terms: the first term is the state quantity bias term, which measures the deviation between the corrected state quantity and the prior state estimate, and is weighted by the prior covariance matrix; the second term is the sum of squares of the observation residuals of all feature points, weighted by the observation noise covariance matrix. The significance of the objective function is to find an error state quantity such that the corrected state quantity neither deviates too much from the prior estimate nor fails to conform well to the observation data.

[0479] The solution that minimizes the objective function can be obtained through the Kalman filter update step. First, the Kalman gain is calculated, and its formula considers the prior covariance matrix, the observation matrix, and the observation noise covariance matrix. The Kalman gain can be understood as a "transformation coefficient" that maps the observation residuals from the observation space to the state space. Then, the Kalman gain is multiplied by the observation residuals to obtain the correction value of the error state quantity, which is added to the current error state quantity. At the same time, the covariance matrix is ​​updated to reflect the reduction in uncertainty after observation.

[0480] Sub-step S254: Iterative convergence judgment.

[0481] The updated error state variable is injected into the nominal state variable (i.e., the nominal state variable is corrected using the error state variable), and the error state variable is then set to zero. At this point, the state estimate is updated. The iteration stops when the convergence condition is met.

[0482] Specifically, checking convergence conditions includes calculating the changes in the state estimates before and after the current iteration (e.g., changes in position and angle). If the changes are less than a second preset threshold (e.g., a position change of less than 1 mm, an angle change of less than 0.01 degrees), or if the number of iterations reaches a preset maximum (e.g., 10 times), then the iteration is considered to have converged, the iteration stops, and the current state estimate is output as the final pose. Otherwise, sub-step S251 (reprojecting the point cloud onto the world coordinate system) is re-executed using the updated state estimate to begin the next iteration.

[0483] The advantage of iterative updates lies in the greater accuracy of the state variables after each iteration. By re-matching the point cloud to the map based on these more accurate state variables, more precise corresponding map points can be found, resulting in more accurate observation residuals and further optimization of the state variables. This positive feedback mechanism can gradually approach the optimal solution, significantly improving the accuracy of pose estimation.

[0484] Step S26: Construct a 3D map based on the final pose of the self-moving device.

[0485] Using the final pose (i.e., the converged state estimate) output in step S25, the point cloud data of the current frame is transformed into the world coordinate system to construct or update a 3D map. This embodiment uses a 3D probabilistic voxel map as the map representation.

[0486] A 3D probabilistic voxel map divides the environment space into a regular 3D voxel grid, where each voxel is a small cube, for example, with a side length of 5 centimeters. Each voxel stores the following information:

[0487] Occupancy probability: This represents the probability that the voxel is occupied by an obstacle, ranging from 0 to 1. 0 indicates that it is definitely free, 1 indicates that it is definitely occupied, and 0.5 indicates that it is unknown.

[0488] Confidence: A measure of the reliability of accumulated observations, used for subsequent noise filtering.

[0489] Centroid coordinates: If multiple points are accumulated within a voxel, the average coordinates (centroid) of these points are used to represent all points within the voxel.

[0490] Feature type: Based on the geometric features of the point cloud within the voxel, it is labeled as edge feature, planar feature, or unknown.

[0491] Map updates include:

[0492] (1) Coordinate transformation: Using the final pose (rotation matrix and translation vector), each point in the current frame point cloud is transformed from the self-moving device coordinate system to the world coordinate system. This transformation is the same as the projection in step S251.

[0493] (2) Voxel Discretization: For each point after transformation to the world coordinate system, the voxel coordinates of the point are calculated according to the preset map resolution (voxel side length). The calculation method is to divide the coordinates of the point by the voxel side length and then round down. For example, if the coordinates of the point are (1.23, 2.45, 0.67) and the voxel side length is 0.05 meters, then the voxel coordinates are (24, 49, 13).

[0494] (3) Update Occupancy Probability: For each voxel containing an observed point, its occupancy probability is updated using the Bayesian update rule. The core of Bayesian update is to calculate the posterior probability based on the prior probability (the original occupancy probability of the voxel) and the observation probability (the probability that the sensor measures that the voxel has a point). For the stability of numerical calculation, the log-odds representation is usually used. The log probability is defined as L = log(P / (1-P)). Under this representation, Bayesian update simplifies to linear addition: new log probability = old log probability + observation log probability increment. The observation log probability increment is a positive number (e.g., +2.0), indicating that the observation of a point increases the log probability of the voxel being occupied; for voxels identified as blank areas, an increment is subtracted. This representation makes map updates very efficient, requiring only addition and bounding operations.

[0495] (4) Intravoxel point cloud compression: When the number of points accumulated within a voxel exceeds a third preset threshold (e.g., 10), the centroid (average of coordinates) of these points is calculated, and this centroid represents all points within the voxel. Simultaneously, by statistically analyzing the feature types (edge ​​features or planar features) of the points within the voxel, a voting method is used to determine the overall feature type of the voxel. This compression process significantly reduces the map's storage space, changing from storing all points to storing centroids, and also reduces the computational load during subsequent point cloud matching, because matching only requires considering the centroid of each voxel, rather than all original points.

[0496] (5) Confidence Maintenance and Noise Filtering: To suppress erroneous observations caused by multipath interference and edge flypoints in the area array lidar, this embodiment maintains a confidence parameter for each voxel. Confidence can be understood as the cumulative degree to which the voxel has been reliably observed. Specifically, the maintenance method is as follows: whenever a voxel is observed (i.e., a point falls into the voxel), its confidence increases by a fixed value (e.g., +1); when a voxel is not observed for a long period of time (e.g., 30 seconds), its confidence decays at a certain rate (e.g., 0.1 decays per second). When the confidence is lower than a preset confidence threshold (e.g., 3), the map point corresponding to the voxel is considered to be noise or a temporary object, and it is removed from the map. Conversely, voxels with high confidence are regarded as stable environmental structures and have higher weight in subsequent matching. This mechanism allows the map to dynamically adapt to environmental changes; for example, moving vehicles and pedestrians can be gradually forgotten, while maintaining the long-term stability of static structures (such as buildings and trees).

[0497] Through the above steps S21 to S26, this embodiment realizes tight coupling and fusion positioning and mapping of three sensors: area array lidar, IMU and wheel odometer.

[0498] In some embodiments, the application scenarios of automatic lawnmowers have been further optimized. The working environment of automatic lawnmowers is mainly outdoor scenes such as lawns and courtyards, and their point cloud data has the following characteristics: the ground (lawn) point cloud accounts for a high proportion, and the ground is usually planar; there are vertical structures such as trees, flower beds, and buildings; the lawnmower may generate vibration and bumps during operation, resulting in relatively large IMU measurement noise; the working area usually has boundaries (such as the boundary between the lawn and the road).

[0499] In this embodiment, the method further includes preprocessing of the ground point cloud. A ground point cloud filtering step is added before step S25 (point cloud iterative correction). Automatic lawnmowers primarily focus on obstacles above ground (such as rocks, tree roots, pet toys, etc.), resulting in a massive amount of ground point cloud data (ground points can account for more than 50% of the total point cloud data in a flat lawn), and the contribution of ground point clouds to localization is limited (because the ground is planar, it can only provide normal vector constraints). Filtering out ground points can significantly reduce the computational load of feature extraction and map matching, while improving the accuracy of obstacle detection.

[0500] The ground filtering method specifically includes the following steps: First, using the attitude information from the current state estimate, the point cloud is transformed into a temporary coordinate system with the self-moving device as the origin. In this coordinate system, the ground is roughly horizontal. Then, the distribution of the point cloud in the vertical direction (Z-axis) is statistically analyzed. Since the height of the self-moving device is known (e.g., 0.2 meters), points near the ground (e.g., Z-coordinates within the range of -0.1 meters to 0.1 meters) are considered ground candidate points. For the ground candidate points, a planar model is fitted using the Random Sample Consensus (RANSAC) algorithm. The RANSAC algorithm can effectively eliminate the interference of outliers (such as blades of grass, small stones, etc.) and robustly fit the ground plane. The fitted ground plane should satisfy the condition that the normal vector is close to vertically upward (0,0,1). Points within a fourth preset threshold (e.g., 2 cm) of the fitted plane are marked as ground points and removed from subsequent processing. After filtering out ground points, the remaining point cloud mainly consists of obstacle points (trees, flower beds, walls, etc.), which are used for subsequent feature extraction and matching.

[0501] In this embodiment, the method further includes an adaptive feature extraction threshold. In a lawn environment, the point clouds generated by flexible objects such as grass and leaves are often scattered and have drastic curvature changes. If a fixed curvature threshold is used, a large number of unstable edge feature points may be extracted in dense grass areas, leading to matching failures or mismatches. Therefore, this embodiment adopts an adaptive feature extraction threshold strategy.

[0502] The core of adaptive thresholding is to dynamically adjust the curvature threshold based on the complexity of the environment. Specifically, this involves calculating the average curvature or curvature distribution of the remaining point cloud after removing ground points from the current frame's point cloud. In open, flat areas (where the remaining point cloud is relatively small and mainly consists of planar points), a higher curvature threshold (e.g., 0.15) is set to reduce the number of unstable feature points, extracting only the most significant ones. In areas with dense obstacles (where the remaining point cloud is large and the curvature distribution is dispersed), a lower curvature threshold (e.g., 0.08) is set to extract more feature points to improve localization accuracy. Adaptive thresholding allows the algorithm to maintain stable matching performance in different scenarios, avoiding localization degradation when features are sparse and computational waste and the risk of mismatches when features are abundant.

[0503] In this embodiment, the method further includes IMU initialization and online zero-bias estimation. When the automatic lawnmower starts, it needs to remain stationary for a period of time (e.g., 5 seconds) to complete IMU initialization. During the initialization phase, the initial zero bias of the gyroscope and accelerometer is calculated using IMU data from the stationary state: the accelerometer zero bias is the average value measured when stationary minus the gravitational acceleration (9.8 m / s²), because in the stationary state, the only acceleration measured by the accelerometer should be gravitational acceleration; the gyroscope zero bias is the average value of the angular velocity measurements (theoretically it should be 0, but in practice, it has a non-zero value due to manufacturing errors). Accurate estimation of the initial zero bias is crucial to the accuracy of subsequent state prediction; if the zero bias estimation deviation is large, the IMU integral will diverge rapidly.

[0504] During operation, the ESKF algorithm continuously estimates and corrects the IMU bias (the error state vector includes gyroscope bias and accelerometer bias). However, due to the continuous low-frequency vibrations of the lawnmower while driving on the lawn, such as vibrations caused by the wheels rolling over the turf, the bias estimation may be disturbed. Therefore, this embodiment incorporates accelerometer bias and gyroscope bias into the state vector and constrains them through wheel odometry and point cloud observations. Specifically, the velocity observation from the wheel odometry can constrain the accelerometer bias because it affects the velocity integral, while the absolute position observation from the point cloud can constrain the gyroscope bias because it affects the attitude integral, thus affecting the accuracy of the point cloud projection. Through this multi-source constraint, the stability of the bias estimation is guaranteed.

[0505] In this embodiment, the method further includes loop closure detection and global optimization. Although ESKF can provide accurate local pose estimation, cumulative drift may still occur over long-term operation. For example, when a lawnmower returns to its origin after circling a large flower bed, its estimated pose may deviate from the initial pose (loop closure error). This embodiment adds a loop closure detection module to identify situations where the self-moving device revisits a mapped area and performs global pose graph optimization.

[0506] Loop closure detection is based on a 3D probabilistic voxel map: the current frame's point cloud is globally matched with the map. If the matching score (e.g., inlier ratio, average residual, etc.) exceeds a score threshold, a loop closure is considered detected. After loop closure detection, the relative pose constraints between the current frame and historical keyframes (i.e., frames saved during previous mapping) are calculated. All constraints between keyframes (including continuous constraints between consecutive frames and loop closure constraints) are constructed into a pose graph. A nonlinear optimization method (such as the Gauss-Newton method or the Levenberg-Marquardt method) is used for global optimization to optimize the poses of all keyframes, minimizing the overall error of all constraints. After optimization, the coordinates of all points in the map are adjusted accordingly to ensure global consistency across the entire map. Loop closure detection and global optimization are typically performed in a background thread, without affecting the frame rate of the front-end real-time localization.

[0507] In this embodiment, the method further includes map saving and reuse. Automatic lawnmowers typically operate repeatedly within a fixed work area (e.g., mowing once a week). This embodiment supports saving the completed 3D map to local non-volatile memory (such as Flash storage or an SD card). Map data includes: the location (voxel coordinates), occupancy probability (logarithmic probability form), centroid coordinates, feature type, confidence level, and other information for each voxel.

[0508] For the next task, the user can load an existing map via a mobile app or automatic triggering. After loading the map, the mobile device does not need to rebuild the map; it can be directly located within the map (only steps S21 to S25 are executed, skipping the map building part of step S26). In positioning mode, the algorithm only needs to estimate the pose of the mobile device in the existing map, further reducing the computational load while achieving higher positioning accuracy because the map is a pre-built high-precision map. This method is particularly suitable for repetitive tasks in fixed areas, saving mapping time before each task.

[0509] During actual operation, the automatic lawnmower moves along a pre-planned path or a real-time generated path to perform mowing. Simultaneously, a solid-state area array LiDAR continuously collects environmental point clouds, an inertial measurement unit collects motion information, and a wheeled odometer collects wheel speed information. Based on the multi-source data fusion positioning, the precise pose of the automatic lawnmower is calculated in real time, and a 3D map of the operating environment is constructed. The constructed 3D map can be used for the following tasks:

[0510] Boundary recognition: Identify the boundaries of the mowing area by the intersection of ground and non-ground areas in the point cloud (such as lawn and flower bed, lawn and road surface) to prevent the lawnmower from driving out of the working area;

[0511] Obstacle detection: Identifies protruding objects above the ground (stones, tree roots, fallen branches, pet toys, etc.) and controls the lawnmower to go around or stop, protecting the blades and self-moving equipment;

[0512] Path planning: Based on map information, plan efficient mowing paths (such as bow-shaped coverage paths) to ensure full coverage without omissions;

[0513] Location and recharging: When the battery is low, the system uses the map and current position to plan a path back to the charging station.

[0514] The control method for the self-moving device in this application achieves the following technical effects through tight coupling and fusion of a non-scanning single-frame pulse imaging sensor, an inertial measurement unit, and a wheel-type odometer:

[0515] (1) Motion distortion correction: High-frequency data from the inertial measurement unit is used to correct motion distortion in the point cloud acquired by the non-scanning single-frame pulse imaging sensor, effectively eliminating the influence of the self-moving device's motion on the point cloud quality, making subsequent point cloud matching more accurate. Compared with schemes that do not perform distortion correction or only use uniform velocity model approximation correction, the correction accuracy of this application is higher, especially suitable for scenarios involving bumps and vibrations generated when an automatic lawnmower travels on the lawn.

[0516] (2) Tightly Coupled Fusion: An error-state Kalman filter framework is used to tightly couple and fuse the data from the three sensors. Compared to the loosely coupled scheme (processing each sensor's data separately before fusing), the tightly coupled scheme can better utilize the complementary characteristics between the sensors: the high-frequency data from the inertial measurement unit provides continuous pose prediction, the velocity data from the wheel odometer provides constraints to suppress the integral drift of the inertial measurement unit, and the point cloud data from the non-scanning single-frame pulse imaging sensor provides absolute environmental observation to correct accumulated errors. The three form a positive feedback loop, achieving the effect of "1+1+1>3".

[0517] (3) Iterative refinement mechanism: An iterative update method is adopted in the point cloud matching stage. After each update, the point cloud to map is rematched using more accurate state variables, gradually approaching the optimal solution. This iterative mechanism significantly improves the accuracy of pose estimation, especially in feature-sparse environments.

[0518] (4) Noise Suppression and Map Maintenance: By classifying feature points (edges / planes) and weighting map point confidence, measurement noise caused by multipath interference and edge flying points in the area array lidar is effectively suppressed. At the same time, the map confidence decay and elimination mechanism enables the map to be updated dynamically, eliminating the long-term effects of dynamic objects and erroneous observations.

[0519] (5) Strong applicability: This application does not rely on Global Navigation Satellite System (GNSS) signals and can work stably in various environments such as indoors, outdoors, under trees, and next to buildings. It is particularly suitable for consumer-grade mobile self-moving devices such as automatic lawnmowers. The constructed 3D map can provide rich environmental information for tasks such as path planning, boundary recognition, and obstacle avoidance.

[0520] In some embodiments, the control method for the self-moving device further includes: acquiring global map information; and acquiring global positioning information based on 3D point cloud data, motion state data, and global map information, i.e., repositioning.

[0521] In a specific embodiment, as shown in Figure 21, when the self-mobile device needs to reposition itself on the map, it first loads an existing map and creates a repositioning map; it acquires a repositioning point cloud using an area array lidar, and then filters and sparsifies the repositioning point cloud data to obtain uniformly distributed point cloud data within the field of view; based on the created repositioning map and the uniformly distributed point cloud data within the field of view, the self-mobile device performs a global positioning search based on a feature-based or pyramid-based global positioning algorithm, which can be the 3D-BBS algorithm and its improved algorithms or other similar algorithms, and outputs global positioning data to achieve the repositioning function; for example, the 3D-BBS algorithm is a global positioning method for 3D point cloud scanning matching, which extends the existing branch-and-bound (BnB) based 2D scan matching algorithm (BBS). 3D-BBS aims to improve the speed and accuracy of global positioning in 3D space; the specific algorithm details are well-known in the art and will not be elaborated here.

[0522] In a further embodiment, to improve positioning accuracy, the self-moving device further includes a motion control system, taking an IMU as an example, used to acquire the pose data of the self-moving device. Based on the created relocalization map, extracted point cloud data uniformly distributed within the field of view, and the pose data of the self-moving device, the self-moving device performs a global positioning search based on feature-based or pyramid-based global positioning algorithms, outputting global positioning data to achieve the relocalization function. Since the pose can be determined through the IMU, the computation of 3D data is reduced to 2D data, effectively achieving dimensionality reduction and helping to improve the data processing speed.

[0523] Referring to Figure 22, to solve the above problems, this application also provides a self-moving device 10, including: a moving module 110 for driving the self-moving device 10 to move; a working module 120 for performing working tasks; a control module 130 for controlling the operation of the moving module 110 and the working module 120; a battery module 140 for providing power to the moving module 110, the working module 120, and the control module 130; and the aforementioned sensor 150. The control module 130 includes a memory and a processor. The memory stores a computer program executable by the processor. When the processor executes the computer program, it implements the aforementioned control method for the self-moving device. The sensor 150 is communicatively connected to the control module 130. Taking an automatic lawnmower as an example, the moving module 110 is the self-driving module of the automatic lawnmower, including wheels and a self-driving motor for driving the wheels to move. The working module 120 is the mowing module, including cutting blades and a mowing motor for driving the cutting blades to rotate. The control module 130 connects to and controls the self-driving motor and the mowing motor.

[0524] In some embodiments, as shown in FIG23, the self-moving device includes a detection module, a processing module, an execution entity, and a communication module. The processing module is connected to the detection module, the communication module, and the execution entity, respectively. Specifically, the detection module is a non-scanning single-frame pulse imaging sensor, and the processing module outputs indication information to the communication module and / or the execution entity based on the target information. The communication module supports communication protocols such as Wi-Fi, 4G, and 5G, and is used to transmit data to an APP or the cloud for users to view and process at any time. The execution entity includes any execution module that can respond to the needs of the self-moving device, such as a motor or a display module.

[0525] Referring to FIG24, in order to solve the above problems, this application also provides an electronic device 300, including a memory 302 and a processor 301. The memory 302 stores a computer program that can be executed by the processor 301. When the processor 301 executes the computer program, it implements the above-mentioned control method for the self-moving device.

[0526] To address the aforementioned problems, this application also provides a computer-readable storage medium storing a processor-executable computer program configured to cause the processor to implement the aforementioned self-moving device control method when executing the computer program.

[0527] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0528] The control method, self-moving device, electronic device, and computer-readable medium of this application can improve three-dimensional perception performance. Compared with traditional single-line radar solutions, the obstacle detection dimension is expanded from a two-dimensional plane to three-dimensional space, and the accuracy of low obstacle recognition is significantly improved, especially for low obstacles. The elevation modeling accuracy based on three-dimensional point clouds is significantly improved, and combined with the motion control system, pitch angle adaptive adjustment is achieved, effectively reducing the risk of overturning in complex terrain. Through the above solutions, the system integration is enhanced. By replacing the traditional multi-sensor fusion solution with a single detection module, the hardware cost is significantly reduced, and the power consumption is significantly reduced. At the same time, it also has extended functions, which can not only realize the recognition of various road surface information, weather information, etc., but also ensure that the self-moving device can adapt to various complex environments and improve environmental adaptability.

[0529] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. A control method for a self-moving device, characterized in that, The self-moving device includes a non-scanning single-frame pulse imaging sensor, which is configured to acquire information about the surrounding environment of the self-moving device and obtain three-dimensional point cloud data. The control method for the self-moving device includes: Acquire the three-dimensional point cloud data; Based on the three-dimensional point cloud data, determine the target information; The target information includes at least one of the following: target attributes, environmental terrain, weather conditions, and sensor operating environment conditions.

2. The control method for a self-moving device according to claim 1, characterized in that, Also includes: Based on the target information, the operating status of the self-moving device is controlled; The operating state of the self-moving device includes the motion state of the self-moving device and / or the working state of the self-moving device.

3. The control method for a self-moving device according to claim 1, characterized in that, The step of determining the target information based on the three-dimensional point cloud data includes: Based on the three-dimensional point cloud data, determine the target feature parameters; The target information is determined based on the target feature parameters.

4. The control method for a self-moving device according to claim 3, characterized in that, The step of determining the target information based on the target feature parameters includes: The target information is determined based on whether the target feature parameters meet the preset target conditions.

5. The control method for a self-moving device according to claim 1, characterized in that, The step of determining the target information based on the three-dimensional point cloud data includes: Based on preset feature conditions, feature points are extracted from the three-dimensional point cloud data to obtain candidate point set data; The target information is determined based on the candidate point set data; The target information includes at least one of the following: target attributes and environmental terrain.

6. The control method for a self-moving device according to claim 1, characterized in that, Also includes: In response to the satisfaction of preset detection conditions, the target information is determined based on the three-dimensional point cloud data.

7. The control method for a self-moving device according to claim 1, characterized in that, The step of determining the target information based on the 3D point cloud data is configured as follows: The target information is determined solely based on the three-dimensional point cloud data.

8. The control method for a self-moving device according to claim 1, characterized in that, The target information includes at least two of the following: target attributes, environmental terrain, weather conditions, and sensor operating environment conditions.

9. The control method for a self-moving device according to claim 1, characterized in that, The non-scanning single-frame pulse imaging sensor is positioned in the direction of travel of the self-moving device to collect environmental information in front of the self-moving device.

10. The control method for a self-moving device according to claim 1, characterized in that, The non-scanning single-frame pulse imaging sensor has a horizontal field of view of 75° to 150° and a vertical field of view of 15° to 90°.

11. The control method for a self-moving device according to claim 1, characterized in that, The target attributes include at least one of obstacles, passable areas, and charging piles; wherein, the obstacles include at least one of static obstacles and dynamic obstacles; And / or, the environmental terrain includes at least one of cliffs and slopes; And / or, the weather condition includes at least one of rainy weather and foggy weather; And / or, the sensor operating environment status includes the sensor's occlusion status.

12. A control method for a self-moving device, characterized in that, The self-moving device includes: A non-scanning single-frame pulse imaging sensor is configured to acquire environmental information about the self-moving device to obtain three-dimensional point cloud data; and The state detection unit is configured to detect the motion state data of the self-moving device; The control method for the self-moving device includes: Acquire the 3D point cloud data and the motion state data; Align the 3D point cloud data and the motion state data according to the timestamp to obtain the 3D point cloud data and the motion state data at the same moment; The three-dimensional point cloud data and the motion state data at the same moment are projected onto the same coordinate system to obtain multi-source data at the same moment and in the same coordinate system; Based on the multi-source data, the pose information of the self-moving device is calculated and / or an environmental point cloud map is constructed.

13. The control method for a self-moving device according to claim 12, characterized in that, The state detection unit includes an inertial measurement unit; the motion state data includes acceleration and angular velocity. And / or, the state detection unit includes a wheeled odometer; the motion state data also includes displacement and orientation angle.

14. The control method for a self-moving device according to claim 12, characterized in that, The step of calculating the pose information of the self-moving device based on the multi-source data, and / or constructing an environmental point cloud map, includes: Based on the multi-source data, a state estimation is performed using a fusion algorithm to output the pose information of the self-moving device and / or an environmental point cloud map.

15. The control method for a self-moving device according to claim 12, characterized in that, Also includes: Get global map information; Global positioning information is obtained based on the 3D point cloud data, the motion state data, and the global map information.

16. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the control method for the self-moving device according to any one of claims 1 to 15.

Images