Cliff detection method and device, mobile robot, and computer-readable storage medium

Abstract

This invention relates to a cliff detection method. First, N×M sets of distance data are collected using a matrix depth sensor, converted into pixel height coordinates in the robot's body coordinate system, and invalid pixels are uniformly assigned values. Then, an m×m detection sub-region is extracted from the data matrix, and multiple adjacent pixel sub-regions are divided using a k×k window with no step size. Two judgment rules are then executed in parallel. Based on the geometric low-position criterion, the number of valid pixels below a ground threshold is counted; those meeting the threshold are included in the danger candidate set. Based on the invalid density criterion, the number of invalid pixels is counted; those meeting the threshold are included in the signal loss candidate set. The two candidate sets are combined to classify multi-level cliff states. This method effectively avoids false detection problems caused by single-point noise. This invention also discloses a cliff detection device, a mobile robot, and a computer-readable storage medium.

Description

Technical Field

[0001] This invention relates to the field of mobile robot safety detection technology, specifically to a cliff detection method, device, mobile robot, electronic device, and computer-readable storage medium. Background Technology

[0002] Mobile robots frequently encounter fall-prone surfaces such as the edges of descending stairs, elevator shafts, and loading / unloading platforms during their movement. A common characteristic of these hazardous surfaces is that, in the robot's coordinate system, the ground height coordinates of this area are significantly lower than the normal wheel contact surface. Current technologies primarily employ the following methods for detecting such cliff or fall risks: 1) Single-point photoelectric cliff sensor: Multiple reflective infrared or single-beam ToF sensors are installed on the leading edge of the robot's bottom, each corresponding to a fixed footprint point. This solution has a small number of sensors, limited coverage width, and is prone to false alarms on low-reflectivity surfaces such as dark carpets; 2) Cliff detection based on depth cameras: This method uses RGB-D or structured light cameras to capture depth images of the front. However, this approach is power-consuming and bulky, and is sensitive to ambient light, making it unsuitable for small mobile robots where cost and power consumption are critical. 3) Simple thresholding method: Directly apply a height threshold to each pixel in the depth matrix, or treat any invalid pixel as a cliff trigger condition. However, real cliff edges often have transition pixels, and a single threshold judgment may miss them due to averaging; sporadic invalid pixels on low-reflectivity ground or ground edges due to signal loss, if processed according to the logic of "any invalid pixel triggers an alarm", are very likely to cause a large number of false alarms; pixels in distant non-monitoring directions due to the wide field of view of the matrix sensor are also easily misjudged as fall zones; 4) In scenarios with strong reflection / transmission, such as mirrors, polished surfaces, or glass, depth sensors often output invalid values ​​because they cannot receive valid echoes. Existing technologies cannot effectively distinguish this "signal loss" from the actual "low ground height," thus leading to missed detections.

[0003] Therefore, existing cliff detection methods lack a unified and effective processing mechanism when facing typical scenarios such as mirror / low-reflection cliffs, transition zones at the edge of a fall, and the difficulty in distinguishing between sporadic invalid pixels and lost real signals within the sensor. Their reliability and anti-interference capabilities need to be improved. Summary of the Invention

[0004] The purpose of this invention is to provide a cliff detection method (and device, mobile robot, electronic device, computer-readable storage medium) to solve the technical problems of insufficient compatibility and anti-interference ability of the prior art when detecting various types of cliffs.

[0005] To achieve the above technical solution, the technical solution of the present invention is as follows: A cliff detection method, comprising: acquiring N×M distance / depth readings collected by a matrix depth sensor covering the ground in the direction monitored by a robot, wherein N and M are both not less than 3; projecting the readings onto the robot body coordinate system to obtain the height coordinates corresponding to each pixel, wherein when a pixel is invalid, its height coordinates are set to a preset invalid value; extracting an m×m sub-region located in the monitoring direction from the N×M matrix, wherein m is less than the smaller of N and M and not less than 3; performing stepless sliding division on the m×m sub-region with a k×k window to obtain multiple spatially adjacent pixel sub-regions, wherein k is less than m and not less than 2; applying the following two independent judgment criteria to each pixel sub-region: (a) Geometric low position criterion: counting the number of pixels in the sub-region whose height coordinates are not the invalid value and are less than a predetermined ground threshold, and when the number of pixels is not less than a first threshold, recording the sub-region. (a) Dangerous candidate set, and save all pixels in the sub-region that pass the geometric low-bit condition; (b) Invalid density criterion: count the number of pixels in the sub-region whose height coordinates are invalid values, and when the number of pixels is not less than the second threshold, record the sub-region in the signal loss candidate set; output a multi-level cliff state according to the dangerous candidate set and the signal loss candidate set, including: (i) if the dangerous candidate set is not empty, and the minimum longitudinal distance of all pixels in the set relative to the robot along the leading edge of the monitoring direction is less than the first distance threshold, then output the first danger level; (ii) if the dangerous candidate set is not empty, and the minimum longitudinal distance is not less than the first distance threshold but less than the second distance threshold, then output the second danger level; (iii) if the dangerous candidate set is empty and the signal loss candidate set is not empty, then output the second danger level; (iv) if both the dangerous candidate set and the signal loss candidate set are empty, then output no danger level.

[0006] Furthermore, the matrix depth sensor can be any of the following types: time-of-flight matrix sensor, structured light depth sensor, binocular stereo depth sensor, or a composite sensing module that generates an equivalent N×M distance array by observing and projecting single-line or multi-line lidar onto the robot's body and then meshing it according to a preset grid.

[0007] In one implementation, the monitoring direction is any one of the following: directly in front of the robot, directly behind it, to the side, or in a preset direction.

[0008] Furthermore, the first threshold is not less than round(2· / 3), the second threshold is not less than round(8) / 9), where round(·) represents the rounding operation.

[0009] In one implementation, when N=M=8, m=4, and k=3, the first threshold is 6 and the second threshold is 8.

[0010] Furthermore, the predetermined ground threshold is a negative value between 0.02m and 0.10m below the ground reference surface, the first distance threshold is no greater than 0.6m, and the second distance threshold is no greater than 2.0m.

[0011] As one implementation, the step of outputting multi-level cliff states further includes: maintaining a state history of multiple consecutive frames, applying a minimum consecutive trigger frame limit to the output that jumps directly from the no-danger level to the first danger level, and outputting the second danger level if the limit is not reached; and the method further includes outputting a set of pixel indexes of the trigger sub-region.

[0012] Furthermore, the method further includes: obtaining the current pitch angle of the robot body; when the absolute value of the pitch angle is greater than the attitude lower limit threshold, dynamically correcting the predetermined ground threshold in the geometric low position criterion, the correction formula being: ; In the formula, Let be the longitudinal distance of the i-th pixel relative to the leading edge of the robot's monitoring system. The static reference ground threshold; Let the pitch angle be the angle when When the threshold is less than or equal to the attitude lower limit threshold, the static reference ground threshold is used directly; when When the value exceeds the preset attitude upper limit threshold, all judgment outputs in step S4 are blocked.

[0013] As one implementation, the lower attitude threshold is 2°; and when the absolute value of the pitch angle is greater than the upper attitude threshold, the output of the cliff state is blocked, and the upper attitude threshold is 15°.

[0014] Furthermore, the method also includes: receiving the trigger state output by at least one single-point cliff sensor installed on the robot body; and fusing the multi-level cliff state with the trigger state according to a preset rule to obtain the final output state.

[0015] As one implementation method, the preset rules include: when the multi-level cliff state is at the first danger level and the trigger state indicates that a cliff has been triggered, the final output state is at the first danger level and is accompanied by a high confidence flag; when the multi-level cliff state is at the second danger level and the trigger state indicates that a cliff has been triggered, the final output state is at the first danger level; when the multi-level cliff state is at the no-danger level and the trigger state indicates that a cliff has been triggered, the final output state is at the second danger level and is accompanied by a flag indicating that a secondary confirmation is required; when the multi-level cliff state is at either the first danger level or the second danger level, and the trigger state indicates that the state is normal, the final output state maintains the original multi-level cliff state and is accompanied by a suspicious flag.

[0016] Furthermore, the triggering state participating in the fusion must satisfy the requirement that the time difference with the current fusion time does not exceed the preset maximum delay; otherwise, it is considered an invalid input and output according to the original multi-level cliff state.

[0017] As one implementation, the method further includes online zero-point correction for temperature drift: under the condition that the robot is stationary and the proportion of pixels with height coordinates other than the invalid value in the current frame is not less than a preset ratio, correction is triggered; the robust median of the height coordinates of the pixels with height coordinates other than the invalid value in the m×m sub-region is used as an instantaneous estimate of the zero-point offset for the current frame; the instantaneous estimate is updated by a moving average to obtain the updated zero-point offset; the predetermined ground threshold used for the geometric low-position criterion in subsequent frames is determined by the sum of the static ground threshold and the updated zero-point offset.

[0018] Furthermore, the robust median is the median of the height coordinate sequence; the formula for updating the moving average is z_bias←(1 α)·z_bias+α·z_bias_inst, where: z_bias is the updated zero-point bias, z_bias_inst is the instantaneous estimate, and α is the preset smoothing coefficient.

[0019] This invention also provides a cliff detection device, comprising: a data acquisition module for acquiring N×M distance / depth readings of the ground covering the robot's monitoring direction, collected by a matrix depth sensor; a projection module for projecting the readings onto the robot's body coordinate system to obtain the height coordinates corresponding to each pixel, wherein when a pixel is invalid, its height coordinates are set to a preset invalid value; a monitoring direction extraction module for extracting an m×m sub-region located in the monitoring direction from the N×M matrix; a sliding window module for performing stepless sliding division on the m×m sub-region using a k×k window to obtain multiple spatially adjacent pixel sub-regions; a dual-criteria determination module for applying a geometric low-position criterion and an invalid density criterion to each pixel sub-region, and generating a dangerous candidate set and a signal loss candidate set; and a risk classification module for outputting a multi-level cliff state based on the dangerous candidate set and the signal loss candidate set; wherein the device is configured to execute the cliff detection method described above.

[0020] The present invention also provides a mobile robot, comprising: a mobile chassis; a matrix depth sensor mounted on the mobile chassis and having its field of view covering the ground in the monitoring direction of the robot; and a processing unit connected to the matrix depth sensor; wherein the processing unit is configured to perform the cliff detection method as described above.

[0021] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0022] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0023] In the cliff detection method, apparatus, mobile robot, electronic device, and computer-readable storage medium provided in this application, an m×m sub-region located in the monitoring direction is first extracted from the projection of N×M readings obtained from the matrix depth sensor. This avoids misjudgment of pixels in non-monitoring directions caused by a wide field of view. Subsequently, the m×m sub-region is divided by sliding a k×k window, and two independent judgment criteria, geometric low position and invalid density, are applied to each pixel sub-region. The geometric low position criterion requires that the number of effective pixels in the sub-region whose height coordinates are below a certain threshold reaches a certain standard (first threshold). This ensures that cliffs with "real ground too low" can be reliably detected and effectively suppresses false triggering by single-point noise. The invalid density criterion requires that the number of invalid pixels in the sub-region (such as signal loss due to specular reflection or exceeding the measurement range) reaches another higher standard (second threshold). This is specifically used to cover "signal loss type" cliffs that are difficult to handle by traditional methods, such as those with mirrors or glass, and also avoids interference from sporadic invalid pixels. Based on the "OR" relationship between these two criteria, this method can simultaneously handle conventional cliffs and "invisible" cliffs made of special materials. Finally, based on the candidate sets of hazards and signal loss, a multi-level state including no hazard, warning, and hazard is output, providing a clear and hierarchical decision-making basis for the upper-level control system. Compared with the prior art, the present invention has the following beneficial effects: 1) Covering mirror / glass cliffs: The invalid density criterion independently captures “signal loss type” cliffs that traditional thresholding methods cannot handle.

[0024] 2) Anti-single-point noise: Both criteria require that the number of pixels that meet the conditions in the pixel sub-region reach a certain threshold, and single-point noise or a small number of sporadic invalid pixels will not trigger an alarm.

[0025] 3) Low computational complexity: The amount of computation is fixed, making it suitable for real-time processing on MCU-level embedded platforms.

[0026] 4) Risk classification: Output three levels of status to provide a clear interface for upper-level control logic.

[0027] 5) Multi-sensor fusion: It can combine IMU pitch angle for dynamic threshold correction to deal with slope scenarios, and can fuse single-point cliff sensors to provide redundant coverage and higher confidence. Attached Figure Description

[0028] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0029] Figure 1 This is a flowchart illustrating a cliff detection method provided in an embodiment of this application.

[0030] Figure 2 This is a schematic diagram illustrating the geometric relationship between the matrix depth sensor and the robot's body coordinate system in an embodiment of this application.

[0031] Figure 3 This is a schematic diagram of m×m sub-region extraction in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Before describing the specific implementation methods, the key terms used in this document will be explained: Preset invalid value: This refers to a standardized numerical value used to identify invalid pixels. When a pixel is marked as invalid by hardware due to lack of valid echo, exceeding the range, low confidence, or other reasons, its height coordinate is set to this value to distinguish it from valid depth measurements. In computer implementations, this value can be NaN (Not a Number) as defined by the IEEE 754 standard, or it can be other conventional special values ​​(such as an extremely large or small number).

[0035] Monitoring direction: This refers to the spatial direction in which the robot needs to focus its cliff detection. This direction is usually the direction in which the robot's mobile platform is traveling (e.g., directly in front), but it can also be configured to other directions (e.g., directly behind, to the side) depending on the application scenario.

[0036] m×m sub-region: refers to a square region with a size of m rows and m columns extracted from the original N×M depth matrix according to the monitoring direction. This region contains the most relevant depth information in the monitoring direction.

[0037] Pixel subregion: refers to a smaller region consisting of k×k spatially adjacent pixels, which is divided by sliding a k×k window on an m×m subregion. It is the basic unit for performing dual-criteria determination.

[0038] Geometric Low-Level Criterion: This refers to a rule that statistically determines whether the height coordinates of effective pixels within a pixel sub-region are below a predetermined ground threshold. This criterion is primarily for conventional cliffs that physically exist but whose height is significantly reduced.

[0039] Invalid density criterion: This refers to the rules for statistically determining the number of invalid pixels (i.e., pixels with preset invalid height coordinates) within a pixel sub-region. This criterion is mainly for special types of cliffs where the signal is completely lost due to specular reflection, transparent materials, or exceeding the measurement range.

[0040] Dangerous candidate set (D): refers to the set of pixel sub-regions that are considered to have "geometric low" cliff risk based on the geometric low criterion.

[0041] Signal loss candidate set (N): refers to the set of pixel sub-regions that are considered to have a "signal loss" cliff risk based on the invalid density criterion.

[0042] Multi-level cliff status: refers to the graded risk assessment results output by comprehensively considering the hazard candidate set D and the signal loss candidate set N. Typically, it includes the hazard level (None), the second hazard level (Warning), and the first hazard level (Danger).

[0043] Please see Figure 1 The diagram shows a flowchart of a cliff detection method provided in an embodiment of this application. This application provides a cliff detection method comprising the following steps S1 to S6: S1. Acquire N×M distance / depth readings of the ground covering the robot's monitoring direction, collected by the matrix depth sensor, where N and M are both not less than 3; S2. Project the reading onto the robot's body coordinate system to obtain the height coordinates of each pixel. When a pixel is invalid, its height coordinates are set to a preset invalid value. S3. Extract an m×m sub-region located in the monitoring direction from the N×M matrix, where m is less than the smaller of N and M and not less than 3; S4. On the m×m sub-region, perform stepless sliding division with a k×k window to obtain multiple spatially adjacent pixel sub-regions, where k is less than m and not less than 2; S5. For each pixel sub-region, apply the following two independent judgment criteria: (a) Geometric low position criterion: Count the number of pixels in the sub-region whose height coordinates are not the invalid value and are less than the predetermined ground threshold. When the number of pixels is not less than the first threshold, the sub-region is recorded in the dangerous candidate set D, and all pixels in the sub-region that pass the geometric low position condition are saved. (b) Invalid density criterion: Count the number of pixels with invalid height coordinates in the sub-region. When the number of pixels is not less than the second threshold, the sub-region is added to the signal loss candidate set N. S6. Based on the hazard candidate set D and the signal loss candidate set N, output the multi-level cliff state, including: (i) If the dangerous candidate set D is not empty, and the minimum longitudinal distance d_min of all pixels in the set relative to the robot's leading edge along the monitoring direction is less than the first distance threshold, then output the first danger level; (ii) If the dangerous candidate set D is not empty, and d_min is not less than the first distance threshold but less than the second distance threshold, then output the second danger level; (iii) If the danger candidate set D is empty and the signal loss candidate set N is not empty, then output the second danger level; (iv) If both the danger candidate set D and the signal loss candidate set N are empty, then output no danger level.

[0044] In this embodiment, the above steps work collaboratively. Steps S1 and S2 acquire the raw depth perception data and unify it into the robot's body coordinate system, providing standardized input for subsequent processing. Step S3 extracts an m×m sub-region, focusing the calculation on the most critical monitoring direction, effectively filtering out interference information from non-monitoring directions in the wide field-of-view sensor. Step S4 uses a k×k sliding window operation to further refine the sub-region into multiple local neighborhoods, enabling the detection algorithm to capture spatially continuously distributed cliff features, while providing a spatial basis for subsequent "density" statistics. Step S5 is the core of this method, independently detecting two cliff physical characteristics: "measurable but too low" and "completely unmeasurable," by applying the geometric low-position criterion and the invalid density criterion in parallel. Both criteria require a certain number of pixels within the pixel sub-region to meet the conditions (i.e., the first threshold and the second threshold), which constitutes a spatial consistency check, effectively suppressing false alarms caused by single-point measurement noise, minor ground irregularities, or sporadic sensor failures. Step S6 then synthesizes and classifies the detection results. It not only distinguishes between the presence and absence of danger, but also further quantifies the degree of danger into two levels, "warning" and "danger," by calculating the longitudinal distance d_min between the nearest point to the robot's leading edge in the dangerous pixel. This provides the upper-level control system with a more refined and physically intuitive decision-making basis. Together, these technologies solve the problem that existing technologies cannot reliably detect both conventional cliffs and special cliffs such as mirrors / glass simultaneously, and are susceptible to single-point noise interference, achieving a balance between high detection rate and low false alarm rate.

[0045] Taking a specific application scenario as an example, a household robotic vacuum cleaner has an 8×8 time-of-flight matrix sensor installed in front of it. As the robot moves forward, steps S1 and S2 are executed, acquiring 64 distance readings per frame and projecting them to obtain the height coordinate z_i. Assume there is a staircase edge 0.4 meters in front of the robot. Step S3 extracts a 4×4 sub-region (m=4) centered from the sensor matrix. Step S4 performs a 3×3 (k=3) sliding window operation on this 4×4 sub-region, obtaining 4 pixel sub-regions. For the pixel sub-region covering the staircase edge, the z_i value of most of its effective pixels will be significantly lower than the ground threshold (e.g., -0.05 meters). Step S5 uses the geometric low-order criterion to count the number of pixels in this sub-region with effective z_i values ​​below the threshold, let's say 7. If this exceeds the first threshold (e.g., 6), then this sub-region is added to the danger candidate set D. Simultaneously, the invalid density criterion counts the number of invalid pixels within this sub-region. Since the edge of the staircase is not a mirror surface, the number of invalid pixels may be very small (e.g., 1), not reaching the second threshold (e.g., 8), and therefore not included in set N. Step S6 calculates the minimum vertical distance d_min of all pixels in set D, assuming it is 0.35 meters, which is less than the first distance threshold (e.g., 0.6 meters). Then, the first danger level (Danger) is finally output, triggering the robot's emergency braking to avoid a fall.

[0046] Based on the above embodiments, the matrix depth sensor mentioned above can be any of the following types: time-of-flight matrix sensor, structured light depth sensor, binocular stereo depth sensor, or a composite sensing module that generates an equivalent N×M distance array by projecting observations from single-line or multi-line LiDAR onto the robot's body and meshing it according to a preset grid. Furthermore, combining the above principles, this approach offers flexibility in sensor selection and universality of the algorithm. The processing logic of this method does not depend on a specific type of depth sensor hardware; its core input is only an N×M distance / depth reading matrix and the corresponding invalid flags. As a specific implementation, the time-of-flight matrix sensor can be STMicroelectronics' VL53L7CX or VL53L8CX (8×8 matrix), or VL53L5CX (4×4 matrix). The depth map output by the structured light or binocular stereo depth sensor (such as Intel RealSense D455) can be adapted to an N×M input array through downsampling or cropping. For robots already equipped with single-line lidar for SLAM, cliff detection can be achieved by projecting the laser point cloud onto the ground in the monitoring direction and dividing it into a grid to form an equivalent N×M range array, thereby reusing existing sensors without the need for additional dedicated hardware modules.

[0047] Based on the above embodiments, the monitoring direction mentioned above can be any direction selected from the robot's front, rear, side, or a preset direction. Furthermore, combining the above principles, this approach allows the same detection algorithm to be adapted to various application scenarios such as preventing falls while the robot is moving forward, preventing falls while reversing, and preventing lateral falls along walls or platform edges, thus improving the reusability of the algorithm and the configuration flexibility of the robot system.

[0048] Based on the above embodiments, regarding the first threshold and the second threshold mentioned above, the first threshold is not less than round(2· / 3), the second threshold is not less than round(8) / 9), where round(·) represents the rounding operation. Further, combining the above principles, this formulaic definition can ensure that the threshold setting and the sliding window size k maintain a reasonable proportional relationship, so that the spatial consistency requirement can adapt to different detection granularities. The first threshold requires that about two-thirds of the effective pixels in the sub-region are in the low position, ensuring the reliability of "geometric low position" cliff detection. The second threshold requires that nearly 90% of the pixels in the sub-region fail simultaneously, which is extremely strict and can effectively filter out sporadic invalid pixels caused by sensor noise, edge transition or local low reflection of the ground, thus specifically targeting large-area, continuous signal loss scenarios (such as mirrors, glass) for triggering, significantly reducing false alarms. As a specific implementation method, for the preferred case mentioned above when N=M=8, m=4, k=3, the first threshold is 6 and the second threshold is 8. At this point, the geometric low position criterion requires at least 6 valid pixels in the 3×3 sub-region to be below the ground threshold before triggering; the invalid density criterion requires at least 8 pixels to be invalid before triggering. This means that a single point or a small number of invalid pixels will not cause false alarms, greatly improving the robustness of working in complex ground environments.

[0049] Based on the above embodiments, regarding the aforementioned predetermined ground threshold and distance threshold, the predetermined ground threshold is a negative value between 0.02m and 0.10m below the ground reference surface, the first distance threshold is no greater than 0.6m, and the second distance threshold is no greater than 2.0m. Furthermore, combining the above principles, using this parameter range can balance detection sensitivity and safety. The ground threshold (e.g., -0.03m) defines the ground height drop that can be identified as a "cliff." The first distance threshold (e.g., 0.6m) defines the nearest boundary of the "dangerous" area requiring immediate emergency braking. The second distance threshold (e.g., 1.5m) defines the boundary of the "warning" area requiring an early warning and the initiation of deceleration planning. These parameters have clear physical meanings, facilitating engineering calibration based on robot size, movement speed, and application scenario.

[0050] Based on the above embodiments, regarding step S3 mentioned above, the extracted position of the sub-region is such that it is centered within the sensor's field of view along the monitoring direction, close to the far row of the corresponding ground footprint in the matrix along the monitoring direction, and the footprint of the pixel ray corresponding to the far row of the m×m sub-region on the ground falls 0.3m to 0.6m in front of the robot's leading edge along the monitoring direction. Furthermore, combining the above principles, this extraction strategy ensures that the m×m sub-region of interest covers the critical area closest to the leading edge that the robot is about to enter. Positioning the sub-region near the far row of the sensor matrix is ​​to fully utilize the sensor's detection range, extending the detection "field of view" forward as much as possible, and allowing for a longer reaction time for the robot. Simultaneously, requiring the sub-region to be centered in the monitoring direction avoids the critical area being moved out of the detection range due to sensor installation deviations or non-linear robot movement.

[0051] Based on the above embodiments, step S6, which involves outputting multi-level cliff states, further includes: maintaining a state history for multiple consecutive frames; applying a minimum consecutive trigger frame limit to the output that jumps directly from the no-danger level to the first danger level; and outputting the second danger level if the limit is not met. The method also includes outputting a set of pixel indices for the trigger sub-region. Furthermore, combining the above principles, maintaining state history and limiting the frame count effectively suppresses false triggers caused by transient noise in a single frame from the sensor or brief environmental interference, improving output stability. For example, at least two consecutive frames (T_persist=2) are required to trigger the first danger level before finally outputting "Danger," otherwise, a "Warning" is output. This provides a buffer against momentary false alarms. Simultaneously, the set of pixel indices for the trigger sub-region allows the upper-level system or debugging tools to accurately draw a "cliff heatmap" on a map or visualization interface, intuitively displaying the location and shape of the danger zone, greatly facilitating algorithm debugging, on-site problem analysis, and human-computer interaction.

[0052] Based on the above embodiments, the method further includes: obtaining the current pitch angle of the robot body; when the absolute value of the pitch angle is greater than the attitude lower limit threshold, dynamically correcting the predetermined ground threshold in the geometric low position criterion, the correction formula being: ; In the formula, Let be the longitudinal distance of the i-th pixel relative to the leading edge of the robot's monitoring system. The static reference ground threshold; Let the pitch angle be the angle when When the threshold is less than or equal to the attitude lower limit threshold, the static reference ground threshold is used directly; when When the value exceeds the preset upper limit threshold for attitude, all judgment outputs in step S4 are masked. Furthermore, based on the above principle, this dynamic correction can solve the misjudgment problem when the robot is driving on a slope. When the robot's front wheels lift (positive pitch angle), the observed height z_i of the normal ground in the distance ahead will be negative in its body coordinate system. If a fixed horizontal ground threshold is still used, these normal slope ground points will be misjudged as cliffs. By introducing a pitch angle θ_pitch and dynamically calculating the corresponding ground threshold based on the longitudinal distance x_i of each pixel, a tilted "ground reference surface" is essentially reconstructed in the body coordinate system. The geometric meaning of this formula is that the ground threshold line is tilted synchronously with the robot's pitch attitude, making the judgment benchmark for the slope ground consistent with the real physical ground, thereby avoiding false alarms in slope scenarios. As a specific implementation, regarding the aforementioned lower and upper limit thresholds for attitude, the lower limit threshold is 2°; and when the absolute value of the pitch angle is greater than the upper limit threshold for attitude, the output of the cliff state is masked, and the upper limit threshold for attitude is 15°. Setting a lower threshold (e.g., 2°) is to avoid frequent and unnecessary threshold adjustments due to minor IMU noise or slight road undulations when driving on flat ground, thus maintaining algorithm stability. Setting an upper threshold (e.g., 15°) is because when the slope angle is too large, the sensor's detection geometry may be severely distorted, reducing the reliability of depth measurement. In this case, continuing to output the cliff status may be meaningless or even lead to incorrect decisions, so directly disabling the output is a safer choice.

[0053] Based on the above embodiments, the method further includes: receiving a trigger state output from at least one single-point cliff sensor mounted on the robot body; and fusing the multi-level cliff state with the trigger state according to a preset rule to obtain a final output state. Furthermore, combining the above principles, multi-sensor fusion can leverage the complementarity and redundancy of different sensor technologies to further improve the overall reliability and safety of the system. Matrix depth sensors have a wide coverage area and can detect cliffs of special materials, but may have blind spots or specific false alarms; traditional single-point infrared or ToF cliff sensors are low-cost and technologically mature, but are limited to point coverage and sensitive to reflectivity. Fusing the two can achieve complementary advantages. As a specific implementation method, regarding the aforementioned preset rules, these preset rules include: when the multi-level cliff state is at the first danger level and the trigger state indicates cliff triggering, the final output state is at the first danger level with a high confidence flag; when the multi-level cliff state is at the second danger level and the trigger state indicates cliff triggering, the final output state is at the first danger level; when the multi-level cliff state is at the no-danger level and the trigger state indicates cliff triggering, the final output state is at the second danger level with a flag indicating secondary confirmation is required; when the multi-level cliff state is at either the first or second danger level, and the trigger state indicates normal, the final output state maintains the original multi-level cliff state with a suspicious flag. The rule logic is clear: when both sensors alarm simultaneously, the confidence level is highest, and the most urgent measures (such as emergency stop) can be taken immediately. When only the matrix sensor gives a danger warning, and the single-point sensor does not trigger, the warning is retained but marked as "suspicious," prompting the system to further judge based on other information (such as historical status and environmental models). When only a single-point sensor triggers and the matrix sensor does not detect any abnormality, a "Pending secondary confirmation" warning is output. This can trigger deceleration or require the robot to make slight movements to change the perspective for reconfirmation, rather than stopping immediately. This can effectively suppress isolated false alarms caused by single-point sensors due to dark carpets, etc.

[0054] Based on the above embodiments, regarding the triggering state mentioned above for participating in fusion, the triggering state must satisfy the condition that the time difference with the current fusion time does not exceed the preset maximum delay; otherwise, it is considered an invalid input and output according to the original multi-level cliff state. Furthermore, combining the above principles, using timing constraints can ensure that the data from each sensor participating in the fusion are aligned and valid in time, avoiding fusion decisions based on outdated information due to delays or loss of data from a single sensor, thus ensuring the timeliness and accuracy of the fusion results.

[0055] Based on the above embodiments, the method further includes online zero-point correction for temperature drift: correction is triggered when the robot is stationary and the proportion of pixels with height coordinates other than the invalid value in the current frame is not less than a preset ratio; the robust median of the height coordinates of pixels with height coordinates other than the invalid value in the m×m sub-region is used as an instantaneous estimate of the zero-point offset for the current frame; this instantaneous estimate is updated by a moving average to obtain the updated zero-point offset; the predetermined ground threshold used for the geometric low-position criterion in subsequent frames is determined by the sum of the static ground threshold and the updated zero-point offset. Furthermore, based on the above principles, this online correction can compensate for the systematic measurement drift (zero-point drift) caused by the matrix depth sensor due to long-term operation and changes in ambient temperature. Without correction, drift may cause normally functioning ground to be misjudged as a cliff, or a real cliff to be missed. By selecting a time when the robot is stationary and there are sufficient effective ground observation points, the median of these effective observation heights is used as an estimate of the current "true ground height," thereby calculating the sensor's zero-point deviation. Using the robust median instead of the mean avoids interference from a few outliers (such as small objects on the ground) in the estimation. A smoothing process using a moving average is then applied to obtain a stable and reliable zero-bias estimate, z_bias. Finally, the actual threshold used for judgment becomes z_threshold_base + z_bias, automatically offsetting the sensor's zero-point drift and ensuring long-term stability. As a specific implementation, for the robust median and moving average updates mentioned above, the robust median is the median of the height coordinate sequence; the formula for the moving average update is z_bias ← (1 α)·z_bias+α·z_bias_inst, where: z_bias is the updated zero-point bias, z_bias_inst is the instantaneous estimate, and α is a preset smoothing coefficient. For example, α can be 0.1, indicating that the new instantaneous estimate slowly updates the historical bias value with a weight of % to achieve a smooth transition. The preset ratio R_valid for triggering correction is preferably 0.85, which requires that at least 85% of the pixels in the m×m sub-region are valid to ensure that the ground samples used for estimation are representative.

[0056] It should be noted that in some optional implementations, the parameters m and k can be adjusted accordingly for different sizes of matrix depth sensors (such as 4×4, 16×16, 32×32) to maintain reasonable detection performance. For example, for a 4×4 matrix, m=3 and k=2 can be used, in which case the total number of pixels in the sub-region is 4, the first threshold can be 3, and the second threshold can be 4. For a higher resolution 16×16 matrix, m=6 and k=4 can be used, etc. The sliding window step size can be set to 1 or other values ​​as needed, in addition to no step sliding, but no step sliding can provide the densest detection coverage. When implementing "saving all pixels in the sub-region that pass the geometric low-order condition", the pixel index, coordinates, or direct reference to the original data pointer can be saved to facilitate the calculation of d_min in step S6. The "output set of pixel indexes of the triggered sub-region" can further include the trigger type (geometric low-order or invalid density) to facilitate differentiated visualization rendering in the upper layer. The parameters such as the lower attitude threshold θ_pitch_min, the upper attitude threshold θ_pitch_max, the smoothing coefficient α, the preset ratio R_valid, and the continuous trigger frame limit T_persist can all be optimized and determined in specific products based on experiments. The values ​​given above are only preferred examples and do not constitute a limitation on the scope of protection of this application.

[0057] This application also provides a cliff detection device. The device includes a data acquisition module, a projection module 20, a monitoring direction extraction module 30, a sliding window module 40, a dual-criteria judgment module 50, a risk classification module 60, and a status release module 70. The data acquisition module acquires N×M distance / depth readings of the ground covering the robot's monitoring direction, collected by a matrix depth sensor. The projection module 20 projects these readings onto the robot's body coordinate system to obtain the height coordinates of each pixel, wherein when a pixel is invalid, its height coordinates are set to a preset invalid value. The monitoring direction extraction module 30 extracts an m×m sub-region located in the monitoring direction from the N×M matrix. The sliding window module 40 performs a k×k window-based sliding segmentation on the m×m sub-region without step size, obtaining multiple spatially adjacent pixel sub-regions. The dual-criteria judgment module 50 applies the geometric low-position criterion and the invalid density criterion to each pixel sub-region, generating a danger candidate set D and a signal loss candidate set N. The risk grading module 60 outputs multi-level cliff states based on the hazard candidate set D and the signal loss candidate set N. The state publication module 70 publishes the multi-level cliff states and related information. Optionally, the device also includes an inertial measurement unit interface module, a single-point sensor interface module, a fusion decision module, and a temperature drift correction module to implement the aforementioned functional extensions. These modules work collaboratively to achieve the cliff detection method described above, exhibiting high detection rate, low false alarm rate, and good scene adaptability.

[0058] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the cliff detection method described above.

[0059] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the cliff detection method described in any of the above claims.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art should be able to make equivalent embodiments by making some changes or modifications to the above-disclosed technical content without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A cliff detection method based on a matrix depth sensor, characterized in that, Includes the following steps: S1. Point cloud data acquisition and coordinate system projection: Using a matrix depth sensor installed on the robot body, covering the ground in the direction to be monitored, N×M sets of depth distance readings are collected, where N and M are both not less than 3; based on the sensor extrinsic parameters, all readings are projected onto the robot body's Cartesian coordinate system to obtain a point set P containing N×M three-dimensional points; among them, for invalid pixels with no effective echo, over the range, confidence level below the preset threshold, or abnormal readings, their height coordinates are assigned a preset invalid value. S2. Effective detection sub-region extraction: Extract m×m pixel sub-regions corresponding to the robot's travel direction or preset monitoring direction from the complete N×M depth matrix; where m is a positive integer not greater than the minimum value of N and M and not less than 3; S3. Sliding window segmentation: The m×m pixel sub-region is segmented without step size using a k×k window to obtain (m-k+1). 2 A spatially adjacent pixel sub-region; where k is a positive integer less than m and not less than 2; S4. Dual-criteria sub-region risk assessment: For each pixel sub-region obtained by sliding segmentation, the geometric low-order criterion and invalid density criterion are applied simultaneously for assessment. (a) Geometric low position criterion: count the number of valid pixels n_low whose height coordinates are non-invalid values ​​and whose height is less than the preset ground threshold; when n_low is greater than or equal to the first threshold, it is determined that there is a risk of low position collapse in the sub-region, the sub-region is included in the dangerous candidate set D, and the corresponding valid pixel information is saved. (b) Invalid density criterion: count the number of invalid pixels n_invalid whose height coordinates are preset invalid values ​​in the sub-region; when n_invalid is greater than or equal to the second threshold, it is determined that the sub-region has a risk of signal loss and the sub-region is included in the signal loss candidate set N; S5. Multi-level cliff status output: Based on the distribution of the hazard candidate set D and the signal loss candidate set N, and combined with the minimum longitudinal distance d_min of the candidate pixels relative to the leading edge of the robot monitoring direction, the cliff hazard status is output in stages: (i) If set D is not empty and the minimum vertical distance d_min is less than the first distance threshold, output the first danger level; (ii) If set D is not empty, and the minimum vertical distance d_min is greater than or equal to the first distance threshold and less than the second distance threshold, output the second danger level; (iii) If set D is empty and set N is not empty, output the second danger level; (iv) If both set D and set N are empty, output no danger level.

2. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, The matrix depth sensor can be any of the following sensing devices: (a) Time-of-Flight (ToF) matrix depth sensor; (b) Structured light depth sensor; (c) Binocular stereo depth sensor; (d) Composite sensing module, wherein the composite sensing module collects observation data through single-line or multi-line lidar, projects it onto the robot body coordinate system, and generates an equivalent N×M distance array according to a preset grid rule; When using a Time-of-Flight (ToF) matrix sensor, the matrix dimensions N and M are preferably any one of 4, 8, 16, or 32.

3. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, The preset monitoring direction includes any direction directly in front of, behind, to the left of, or to the right of the robot; Among them, the rear monitoring direction is used for robot reversing and falling detection, while the left and right side monitoring directions are used for robot falling detection when moving along walls and walking along edges. The first threshold value is not less than round(2·k² / 3), and the second threshold value is not less than round(8·k² / 9); in the preferred configuration of N=M=8, m=4, k=3, the first threshold value is 6 and the second threshold value is 8.

4. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, The preset ground threshold is a negative value within the range of 0.02m to 0.10m below the ground reference surface; the first distance threshold is no greater than 0.6m, and the second distance threshold is no greater than 2.0m; In step S2, the m×m sub-regions extracted are arranged centered along the preset monitoring direction and close to the far row of the matrix. The projected footprints of the pixels of the far row of the sub-regions on the ground are located within the range of 0.3m to 0.6m in front of the leading edge of the robot's monitoring direction.

5. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, Step S5 also includes a multi-frame state stabilization constraint and trigger type recording mechanism: Maintain the detection state history of multiple consecutive frames. For sudden scenes that jump directly from no danger level to the first danger level, set a minimum number of consecutive trigger frames. If the frame count condition is not met, force downgrade to the second danger level. Simultaneously record the trigger type and corresponding pixel index of each sub-region. The trigger types include three categories: geometric low-bit trigger, invalid density trigger, and dual trigger. The method also includes a dynamic threshold correction step based on the IMU pitch angle: The robot's real-time pitch angle is collected by the inertial measurement unit. When the absolute value of the pitch angle When the ground threshold exceeds the preset lower limit, the ground threshold is dynamically adjusted pixel by pixel. The adjustment formula is as follows: ; In the formula, Let be the longitudinal distance of the i-th pixel relative to the leading edge of the robot's monitoring system. The static reference ground threshold; when When the threshold is less than or equal to the attitude lower limit threshold, the static reference ground threshold is used directly; when When the value exceeds the preset attitude upper limit threshold, all judgment outputs in step S4 are blocked.

6. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, It also includes a decision-making step that fuses the matrix depth detection status with the status of the single-point reflective cliff sensor: Receive the real-time trigger state S_ir from at least one single-point reflective cliff sensor, fuse the matrix depth detection multi-level state S_tof output from step S5 with S_ir, and output the final cliff detection state S_final. The specific fusion rules are as follows: (a) If S_tof is the first danger level and S_ir triggers the cliff signal, output the first danger level with a high confidence flag; (b) If S_tof is at the second danger level and S_ir triggers the cliff signal, upgrade the output to the first danger level; (c) If S_tof is at no danger level and S_ir triggers the cliff signal, output the second danger level with the pending confirmation mark; (d) If S_tof is of any danger level and S_ir is in a normal, non-triggered state, maintain the original level of S_tof and add a suspicious state flag; (e) In other scenarios, the output state of S_tof is maintained directly; Among them, only data from the same source with a time difference not greater than the preset maximum delay participate in the fusion calculation; when multiple single-point reflective cliff sensors are configured, a unified S_ir state is synthesized through logical OR operation or weighted majority voting algorithm.

7. The cliff detection method based on a matrix depth sensor according to claim 1, characterized in that, It also includes an online zero-point correction step for sensor temperature drift: The robot's operating status is monitored in real time. When the robot is stationary and the proportion of non-invalid pixels in the current frame is not less than a preset threshold, the height coordinates of all non-invalid pixels in the sub-region are extracted, and the instantaneous zero-point bias is calculated using a robust median algorithm. The stable zero-point bias z_bias is obtained by iteratively updating the algorithm through a moving average. The effective ground threshold of all subsequent detection frames is obtained by superimposing the static ground threshold and the real-time zero-point bias z_bias, thereby realizing online compensation for temperature drift error.

8. A cliff detection device, characterized in that, The cliff detection method based on a matrix depth sensor as described in any one of claims 1 to 7 includes: a data acquisition module, a projection module, a monitoring direction extraction module, a sliding window module, a dual-criteria judgment module, a risk classification module, and a status release module. Optionally, it also includes an IMU interface module, a single-point sensor interface module, a fusion decision module, and a temperature drift correction module; each functional module works together to execute all the detection, correction, and status output steps of the method.

9. A mobile robot, characterized in that, Includes a mobile chassis, a matrix depth sensor, and a processing unit; The matrix depth sensor is an N×M matrix depth sensor, which is installed at an angle on the mobile chassis near the preset monitoring direction and the lens is arranged at an angle to the ground. The mobile robot may be equipped with an IMU and at least one single-point reflective cliff sensor; The processing unit is communicatively connected to a matrix depth sensor, an optional IMU, and an optional single-point reflective cliff sensor, and is configured to execute the cliff detection method according to any one of claims 1 to 7; wherein, when the detection outputs a first danger level, the robot is controlled to perform an emergency stop or reverse avoidance action, and when the detection outputs a second danger level, the robot is controlled to perform a deceleration avoidance action.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements all the steps of the cliff detection method based on a matrix depth sensor as described in any one of claims 1 to 7.

Images