Obstacle identification method, robot and storage medium

Abstract

The invention relates to the technical field of image processing, provides an obstacle recognition method, a robot and a storage medium, is applied to the robot, and aims to solve the problems that in the prior art, part of schemes try to improve environmental adaptability through fusion of vision and laser radar, but most of the schemes only stay in simple superposition of single-frame data, and the environmental adaptability is poor. Time sequence utilization of historical frame point clouds is not considered, so that the problem of sparse depth information is still not improved is solved. The method comprises the following steps: acquiring current frame point cloud data and historical adjacent frame point cloud data corresponding to a robot, and acquiring dense point cloud data according to the current frame point cloud data and the historical adjacent frame point cloud data; and acquiring obstacle information according to the dense point cloud data, and completing obstacle recognition of the robot. The method effectively solves the problems of small target leak detection and contour ambiguity caused by single-frame radar point cloud sparsity, and enhances the detection capability of middle and long distance obstacles.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an obstacle recognition method, a robot, and a storage medium. Background Technology

[0002] In obstacle recognition technology for smart lawnmowers, traditional solutions often use binocular vision modules to obtain depth information through parallax calculation. However, binocular vision is highly sensitive to lighting conditions. In complex environments such as low light and strong light, the parallax calculation error increases significantly, leading to inaccurate obstacle distance judgment.

[0003] LiDAR is better adapted to complex environments such as low light and strong light. However, the point cloud of a single-frame LiDAR is naturally sparse at medium and long distances. When projected onto a two-dimensional image plane, it is easy to cause small targets to be missed or the contour recognition to be blurred.

[0004] Therefore, a method is urgently needed to solve the above problems. Summary of the Invention

[0005] This application provides an obstacle recognition method, a robot, and a storage medium, aiming to solve the problem that in existing LiDAR obstacle recognition schemes, small targets are easily missed or contour recognition is blurred due to sparse point cloud depth information.

[0006] In a first aspect, this application provides an obstacle recognition method applied to a robot, the method comprising: Obtain the current frame point cloud data and historical adjacent frame point cloud data corresponding to the robot, and obtain dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data. Obstacle information is obtained based on the dense point cloud data, and obstacle recognition is completed for the robot.

[0007] Secondly, this application also provides a robot, comprising: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the steps of the obstacle recognition method as described in the first aspect above.

[0008] Thirdly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the obstacle recognition method described in the first aspect above.

[0009] This application generates a denser point cloud by fusing point cloud data from the current frame and historical adjacent frames, which effectively compensates for the problem of missed obstacle detection and blurred contours caused by the sparse point cloud of a single-frame LiDAR, and improves the accuracy and robustness of obstacle recognition.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart illustrating the steps of a first obstacle recognition method provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating the steps of a second obstacle recognition method provided in an embodiment of this application; Figure 3 This is a schematic flowchart illustrating the steps of a third obstacle recognition method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an obstacle recognition device provided in an embodiment of this application; Figure 5 This is a schematic block diagram of the structure of a robot provided in one embodiment of this application.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0016] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0017] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0018] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0019] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0020] In obstacle recognition technology for smart lawnmowers, traditional solutions often use binocular vision modules to obtain depth information through parallax calculation. However, binocular vision is highly sensitive to lighting conditions. In complex environments such as low light and strong light, the parallax calculation error increases significantly, leading to inaccurate obstacle distance judgment.

[0021] LiDAR is better adapted to complex environments such as low light and strong light. However, the point cloud of a single-frame LiDAR is naturally sparse at medium and long distances. When projected onto a two-dimensional image plane, it is easy to cause small targets to be missed or the contour recognition to be blurred.

[0022] Therefore, a method is urgently needed to solve the above problems.

[0023] To resolve the above issues, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic flowchart illustrating an obstacle recognition method provided in one embodiment of this application. This obstacle recognition method can be implemented by a robot. This application does not limit the type of robot; a lawnmower is used as an example for illustration.

[0024] To solve the above problem, please refer to Figure 1 Specifically, such as Figure 1 As shown, the provided obstacle recognition method includes steps S101 to S102. Details are as follows: Step S101. Obtain the current frame point cloud data and historical adjacent frame point cloud data corresponding to the robot, and obtain dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data.

[0025] Specifically, the purpose of step S101 is to fuse sparse, single-frame data into a more information-rich point cloud image.

[0026] For example, the current frame point cloud data is the 3D point cloud data of the most recent moment obtained by the robot through real-time scanning of the surrounding environment by the lidar (such as 16-line, 32-line or solid-state lidar) on its device.

[0027] Historical Adjacent Frames Point Cloud (HIPF) is a point cloud dataset retrieved from the robot's memory or cache, containing one or more frames immediately preceding the current frame. Typically, frames that are temporally consecutive and can be effectively aligned using robot motion estimation are selected, such as several frames within the past 100-300 milliseconds.

[0028] Because the robot is constantly moving, the point cloud data collected at different times are in different coordinate systems. Directly superimposing these point clouds will cause ghosting and confusion. Therefore, they must be unified into a single coordinate system.

[0029] By using sensors such as wheel odometry, inertial measurement unit (IMU) or visual odometry built into the robot, the pose change (i.e., rotation and translation matrix) of the robot from the historical frame to the current frame can be accurately calculated.

[0030] Using the calculated pose transformation matrix, the coordinates of each point in the historical frame point cloud are transformed and projected or aligned to the coordinate system of the current frame. For example, if the robot moves forward 0.5 meters, all points in the previous frame need to move backward 0.5 meters in the coordinate system to be in the same spatial reference frame as the points in the current frame. Based on the aligned point cloud, a fusion process is performed to increase the density of the point cloud.

[0031] Voxel grid filtering is an efficient and common method for achieving densification. It divides 3D space into countless tiny cubes (voxels). All aligned point clouds (current frame + historical frames) are then filled into this 3D grid. For each voxel, all 3D points within it are downsampled, for example, by calculating the centroid (average value) of these points, and this centroid is used to represent all points within that voxel.

[0032] Noise reduction eliminates duplicate or floating points within individual voxels caused by sensor noise. By overlaying data from multiple frames, voxels that were originally empty in a single frame are now filled with points from other frames, resulting in higher coverage of the entire point cloud scene, especially compensating for holes caused by sparse point clouds in mid- to long-range regions. Data compression reduces the total number of points, lowering the computational burden of subsequent processing.

[0033] This step effectively transforms time-series information into spatial density information, turning a sparse snapshot at a single moment into a long-exposure photograph accumulated over a short period of time, thus more clearly outlining the contours of the environment.

[0034] Step S102. Obtain obstacle information based on the dense point cloud data to complete obstacle recognition for the robot.

[0035] Specifically, obstacle detection and identification are performed using the high-quality point cloud data produced in step S101.

[0036] Exemplary point cloud preprocessing may include: Ground Segmentation: Using algorithms (such as Ray Ground Filter, RANSAC plane fitting) to separate ground points from non-ground points (obstacle points) in the point cloud. This greatly reduces the number of points that need to be processed and avoids misclassifying ground as obstacles. Clustering: Performing cluster analysis on the non-ground point cloud to divide points that are spatially close into independent sets. A commonly used algorithm is Euclidean clustering. Each cluster represents a potential obstacle.

[0037] For each cluster, its associated obstacle information is calculated, including: Position: The 3D centroid of the cluster points is calculated to obtain the obstacle's coordinates (x, y, z) relative to the robot. Dimension: The bounding box of the cluster in the length, width, and height directions is calculated to understand the size of the obstacle. Contour & Shape: Dense point clouds can more accurately depict the contours of obstacles, which helps in preliminary shape classification (e.g., columnar, cubic, irregular shapes).

[0038] By combining the features of point clouds, even simple classifiers or deep learning models (such as PointNet) can be integrated to perform semantic recognition on clusters, distinguishing between people, animals, trees, toys, or other objects. Because temporal data is used, the system can associate the same obstacle across frames, estimate its speed and trajectory, and achieve dynamic obstacle tracking. This is crucial for robots to make predictive obstacle avoidance decisions.

[0039] Optimized point cloud data is transformed into structured obstacle information that can be understood and utilized by the robot navigation system, providing accurate and reliable input for subsequent path planning and safety decisions.

[0040] In some embodiments, such as Figure 2 As shown, the step of obtaining dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data includes steps S102a to S102c.

[0041] Step S102a. Obtain the current frame pose information and historical adjacent frame pose information corresponding to the robot. Step S102b. Based on the current frame pose information and the pose information of historical adjacent frames, convert the point cloud data of historical adjacent frames to the current frame point cloud coordinate system.

[0042] Step S102c. Obtain the densed point cloud data based on the converted historical adjacent frame point cloud data and the current frame point cloud data.

[0043] This embodiment describes the basic steps for acquiring dense point cloud data. First, the point cloud data of the robot (such as a lawnmower) in the current frame at the current moment and the point cloud data of the immediately preceding frame (i.e., historical adjacent frames) are acquired. Then, based on these two sets of point cloud data, the "dense point cloud data" is finally obtained through a specific processing method.

[0044] The robot's computing unit (such as an embedded industrial computer) continuously reads point cloud data streams from the LiDAR module. The system maintains a data buffer that stores at least the point cloud data of the latest frame (current frame Li) and the previous frame (historical frame Li-1). When obstacle recognition is required, the computing unit calls the "point cloud densification module." The inputs to this module are Li and Li-1, and the output is the fused and densified point cloud data L.

[0045] In some embodiments, the current frame pose information includes current position information and current rotation information, and the historical adjacent frame pose information includes historical position information and historical rotation information; the step of converting the historical adjacent frame point cloud data to the current frame point cloud coordinate system based on the current frame pose information and the historical adjacent frame pose information includes: converting the historical adjacent frame point cloud data according to the current rotation information and the historical rotation information to adjust the rotation posture corresponding to the historical adjacent frame point cloud data to the rotation posture of the current frame; obtaining the position difference between the historical position information and the current position information; and converting the historical adjacent frame point cloud data to the current frame point cloud coordinate system based on the position difference and the converted historical adjacent frame point cloud data.

[0046] This embodiment further refines how to utilize the robot's motion information (pose) to transform the historical point cloud to the current coordinate system. It obtains the current frame pose information (Posei) when the robot collects the current frame point cloud, and the historical adjacent frame pose information (Posei-1) when collecting historical adjacent frame point clouds. Using these two pose information, through mathematical transformation, Li-1 (originally in the historical coordinate system) is transformed to the same current coordinate system as Li. The transformed historical point cloud is then directly superimposed (or fused) with the current frame point cloud to obtain the final densed point cloud L.

[0047] Robots typically integrate inertial measurement units (IMUs), wheel encoders, or visual odometry to estimate their own pose (including position p and rotation quaternion q) in real time. While acquiring point clouds Li and Li-1, the corresponding Posei and Posei-1 are simultaneously recorded. The transformation process follows the principles of rigid body motion in three-dimensional space, eliminating the influence of the robot's own motion on the point cloud position through rotation and translation operations.

[0048] In some embodiments, the step of transforming the historical adjacent frame point cloud data according to the current rotation information and the historical rotation information includes: performing an inverse transformation on the current rotation information; and transforming the historical adjacent frame point cloud data according to the inversely transformed current rotation information and the historical rotation information.

[0049] Specifically, this embodiment focuses on the specific operations of the rotation transformation. In order to align the pose of the point cloud of the historical frame to the current frame, rotation compensation is required. The specific operation is as follows: first, invert the rotation information qi of the current frame (to obtain qi.inverse()), then multiply it with the rotation information qi-1 of the historical frame, and finally multiply the historical frame point cloud Li-1 with this synthesized rotation matrix (or quaternion) to complete the adjustment of the rotation pose.

[0050] The position information p and rotation information q are typically stored as vectors and quaternions, respectively. During computation, corresponding mathematical libraries (such as Eigen) are used to perform quaternion inversion, multiplication, and point cloud rotation operations. This computation is the core of the pose transformation, ensuring that 3D points acquired at different times can be accurately stitched into the same stationary reference frame.

[0051] In some embodiments, such as Figure 3 As shown, the step of obtaining obstacle information based on the dense point cloud data includes steps S102d and S102e.

[0052] Step S102d. Obtain the external lighting conditions corresponding to the robot, and determine the corresponding obstacle recognition algorithm based on the external lighting conditions.

[0053] Step S102e. Obtain the obstacle information based on the obstacle recognition algorithm and the dense point cloud data.

[0054] This embodiment defines how to select different obstacle recognition algorithms based on external lighting conditions. The system first acquires the external lighting conditions, and then selects one of two preset algorithms (normal environment recognition algorithm and low light recognition algorithm) based on these conditions. The selection criterion is based on the average brightness of the real-time environmental image captured by the camera.

[0055] The robot's camera continuously captures images of the environment. The image processing module converts each frame to grayscale and then calculates the average grayscale value of all pixels in the entire image. A preset brightness threshold is used (e.g., set to 50 / 255 through experimental calibration). The program compares the average brightness value with the threshold. If the average brightness value is greater than or equal to the threshold: the lighting is considered sufficient, and the "normal environment recognition algorithm" (i.e., multi-sensor fusion process) is activated. If the average brightness value is less than the threshold: the lighting is considered insufficient, and the "low light recognition algorithm" (i.e., pure point cloud processing process) is activated.

[0056] It should be noted that the camera corresponding to the embodiments of this application can be any type of camera, such as a monocular camera, a binocular camera, a multi-view camera, or a panoramic camera. Those skilled in the art can select any specification of camera according to actual needs to implement the method provided in the embodiments of this application. The embodiments of this application do not limit the type of camera.

[0057] In some embodiments, the obstacle recognition algorithm includes at least a normal environment recognition algorithm and a low-light recognition algorithm; determining the corresponding obstacle recognition algorithm based on the external lighting conditions includes: acquiring a real-time environmental image acquired by an image acquisition device mounted on the robot; performing grayscale processing on the real-time environmental image and calculating the average brightness value corresponding to the grayscale real-time environmental image; comparing the average brightness value with a preset brightness threshold; if the average brightness value is greater than or equal to the brightness threshold, determining the obstacle recognition algorithm as the normal environment recognition algorithm; if the average brightness value is less than the brightness threshold, determining the obstacle recognition algorithm as the low-light recognition algorithm.

[0058] This embodiment describes the action performed by the system in step S102 after the judgment result corresponding to the above embodiment is a normal scene. Based on the lighting conditions, the decision module issues a command to activate (or invoke) a normal environment recognition algorithm specifically designed for well-lit environments. This algorithm is based on a fusion recognition process of image (RGB information) and point cloud (depth information).

[0059] In the ambient light determination module, when the calculated average brightness of the ambient image is greater than or equal to the preset "brightness threshold", the module will switch to a logical flag (for example, setting a boolean variable isNormalLight =True).

[0060] The system's control hub (such as the main scheduler) polls or receives the status of logical flags. Once it detects that the status has become true, it stops any other currently running algorithm processes (for example, if the dark light algorithm was last run, it is safely exited first), and then initializes and runs the normal environment recognition algorithm.

[0061] The initiated normal environment recognition algorithm will take over all subsequent perception tasks. It will simultaneously capture camera images and dense point clouds, and work in the following steps: image preprocessing, dual-path (segmentation + detection) visual obstacle detection, point cloud image projection, depth fusion and decision-making, and formatted output.

[0062] In some embodiments, if the obstacle recognition is a normal environment recognition algorithm, the step of completing obstacle recognition for the robot based on the obstacle recognition algorithm and the dense point cloud data includes: acquiring an image to be recognized collected by the robot; detecting at least one target obstacle in the image to be recognized; projecting the dense point cloud data onto the image coordinate system corresponding to the image to be recognized to obtain depth map information; and outputting an obstacle recognition result based on the target obstacle and the depth map information. The obstacle recognition result includes obstacle information corresponding to each target obstacle, and the obstacle information includes at least one or more of position information, shape information, and distance information.

[0063] This embodiment describes the complete steps of obstacle recognition under normal lighting conditions. First, an image to be recognized (i.e., an RGB image) is acquired using a camera. Then, target detection is performed on this image to identify potential obstacles (target obstacles). Simultaneously, the dense point cloud data obtained in step S101 is projected onto the coordinate system of the image to generate a depth map information corresponding one-to-one with the image pixels. Finally, by combining the detected obstacle regions and their precise depth information, a complete obstacle recognition result containing information such as position, shape, and distance is output.

[0064] In the normal illumination algorithm thread, the camera captures images, and the LiDAR provides a dense point cloud. The images are sent to the downstream detection module, and the point cloud is projected into a depth map by the coordinate transformation module. The fusion decision module receives the detection bounding boxes and the depth map, queries the depth value (which can be the average or median) of the region where each detected obstacle is located, thereby determining its precise 3D position and distance, and packaging it into structured data for output.

[0065] In some embodiments, before detecting at least one potential obstacle in the image to be identified, the method further includes: performing distortion correction on the image to be identified according to preset camera calibration parameters.

[0066] Before analyzing the image, preprocessing is performed to eliminate lens distortion. The original "image to be identified" is corrected using pre-calibrated camera internal parameters (such as focal length, principal point, and distortion coefficients) to obtain a distortion-free image.

[0067] During system initialization, the camera is calibrated using a calibration board such as a checkerboard to obtain and store the camera matrix and distortion coefficients. In real-time processing, for each frame of the original image captured, the calibration parameters are input to obtain the corrected image, which is then used by subsequent deep learning and vision algorithms.

[0068] In some embodiments, the step of detecting at least one target obstacle in the image to be identified includes: performing semantic segmentation on the image to be identified to obtain multiple semantic regions; performing target detection on the image to be identified to obtain at least one potential obstacle and obstacle information corresponding to the potential obstacle; determining a target semantic region corresponding to each potential obstacle in the multiple semantic regions; and determining whether the potential obstacle is the target obstacle based on the obstacle information corresponding to each potential obstacle and the target semantic region.

[0069] This embodiment details how to perform robust obstacle detection on the corrected image. It employs a strategy of parallel semantic segmentation and object detection, with complementary results: Semantic segmentation uses models such as U-Net to perform pixel-level classification of images, outputting a probability map of each pixel belonging to categories such as ground, obstacles, and grass.

[0070] Object detection uses models such as YOLO to locate and classify objects in an image, outputting bounding boxes and categories.

[0071] For each potential obstacle box output by the target detection, the corresponding region (target semantic region) is found in the semantic segmentation result. The two sets of information are combined to determine whether the region is confirmed as the final target obstacle.

[0072] The semantic segmentation network and the object detection network are run in parallel on a computing unit (such as an embedded module equipped with a GPU). For each bounding box output by the detection network, the main semantic labels appearing within the bounding box region are counted on the category mask map output by the segmentation network.

[0073] In some embodiments, determining whether a potential obstacle is the target obstacle based on obstacle information and a target semantic region corresponding to each potential obstacle includes: obtaining semantic label information corresponding to each semantic region; if the semantic label information determines that the target semantic region corresponding to the potential obstacle is not ground, determining that the potential obstacle is the target obstacle; and / or, if the semantic label information determines that the target semantic region corresponding to the potential obstacle is ground, inputting the obstacle information corresponding to the potential obstacle and the semantic label information corresponding to the target semantic region into a preset obstacle recognition model to determine whether the potential obstacle is the target obstacle; and / or, if the semantic label information determines that any semantic region has an obstacle label and no corresponding potential obstacle, inputting the semantic label information corresponding to the semantic region into a preset obstacle recognition model to determine whether the potential obstacle is the target obstacle.

[0074] This embodiment specifies how to use the results of object detection and semantic segmentation to make the final obstacle determination. A three-layer decision logic is set: Strong confirmation: If a region is bounded by object detection as an obstacle (such as a person or a rock), and most pixels of that region are classified as non-ground (such as obstacle category) in the semantic segmentation result, then it is directly determined to be a real obstacle. This is the most reliable evidence. Recall compensation: When the two pieces of information are inconsistent, a complex decision is triggered.

[0075] Scenario A: Only the object is detected as a bounding box, but semantic segmentation identifies it as ground. This could be due to segmentation errors or obstacles being close to the ground. In this case, instead of rejecting the detection, the features of the region (such as the confidence of the bounding box, the texture features of the segmentation map, etc.) are input into a more complex obstacle recognition model (such as a decision tree or random forest) for secondary evaluation.

[0076] Scenario B: The semantic segmentation map shows obvious obstacle pixel regions, but the target detection does not identify them. This may be because the target is too small or the detection model misses it. Similarly, the semantic and image features of this region are input into the obstacle recognition model for judgment. Only areas where the target detection fails to detect and the semantic segmentation does not label them as obstacle regions are considered obstacle-free.

[0077] Implement a fusion decision-maker module. This module receives a list of detection boxes and a segmentation mask image. It iterates and makes judgments according to the rules described above. For cases requiring recall compensation, the system needs to pre-train a classifier (obstacle recognition model) based on multiple manual or deep learning features to handle these ambiguous cases. Finally, it outputs a validated and more reliable list of target obstacles.

[0078] In some embodiments, the robot is equipped with an image acquisition device and a lidar. The lidar is used to acquire the current frame point cloud data, and the image acquisition device is used to acquire the image to be identified. Projecting the dense point cloud data onto the image coordinate system corresponding to the image to be identified includes: obtaining the intrinsic and extrinsic parameter calibration information corresponding to the image acquisition device and the lidar; and projecting the dense point cloud data onto the image coordinate system corresponding to the image to be identified according to the intrinsic and extrinsic parameter calibration information to obtain the depth map information.

[0079] To achieve point cloud and image fusion, the 3D point cloud must be accurately projected onto the 2D image plane. This relies on precise sensor calibration. It requires obtaining the intrinsic and extrinsic parameter calibration information between the camera and the LiDAR (including their respective intrinsic parameters and the extrinsic parameters between them, i.e., the rotation and translation relationship).

[0080] After system assembly, a joint calibration method (such as using a calibration board with special markings) is used to calibrate the camera's intrinsic parameters and the relative pose (extrinsic parameters) of the radar and camera, and these are stored. During real-time processing, for each 3D point P(x, y, z) in the dense point cloud L, the pixel coordinates (u, v) of that point on the image are calculated using the camera's intrinsic parameter matrix K and the radar-to-camera extrinsic parameter transformation matrix T_l2c, according to the formula pixel = K * T_l2c * P. The depth value z' (depth in the camera coordinate system) is then filled into the depth map at the position of (u, v). This generates a depth map with the same size as the image.

[0081] In some embodiments, before outputting the obstacle recognition result based on the target obstacle and the depth map information, the method further includes: obtaining point cloud density information corresponding to each pixel in the image space corresponding to the image to be recognized based on the dense point cloud data; generating quality information based on the point cloud density information, so as to output the obstacle recognition result based on the target obstacle, the depth map information and the quality information.

[0082] Recognizing that not all pixels in the depth map obtained by projection are equally reliable, this embodiment proposes to generate a quality information map (or confidence map).

[0083] Two key metrics are evaluated: Point cloud density: This counts the number of points successfully projected within a small window (e.g., 5x5) around each pixel in the image. Areas with low point density may have depth information derived from interpolation or noise, resulting in low reliability.

[0084] Depth Variation Consistency: Check the depth value of each pixel and its neighboring pixels. If there is a sharp change (such as the depth difference between adjacent pixels exceeding a threshold), the point may be located at the edge of an object or is a noisy point, and it is marked as an "edge / noisy region".

[0085] By combining these analyses, a "quality value" between 0 and 1 is generated for each pixel in the depth map, with a high value indicating a reliable depth. In subsequent fusion, the quality value can be used as a weight, reducing the impact of low-reliability depth information on the final decision.

[0086] After the depth map is generated by projection, a post-processing thread is started. The depth map is traversed, and for each valid depth pixel, density statistics and neighborhood depth difference calculation are performed. The quality value is calculated according to preset rules (e.g., if the density is higher than a threshold and the neighborhood difference is less than a threshold, then the quality = 1.0; otherwise, it is reduced proportionally), and a quality map of the same size as the depth map is generated.

[0087] In some embodiments, before projecting the dense point cloud data onto the image coordinate system corresponding to the image to be identified, the method further includes: obtaining first timestamp information corresponding to the dense point cloud data and second timestamp information corresponding to the image to be identified; and aligning the dense point cloud data and the image to be identified with timestamps according to the first timestamp information and the second timestamp information.

[0088] The camera and LiDAR are two independent sensors, and there may be slight differences in the time points at which they collect data. To ensure the accuracy of data fusion, the data from both needs to be synchronized (timestamp alignment).

[0089] Each frame of image captured by the camera and each frame of point cloud data captured by the LiDAR is precisely timestamped. During the fusion process, the system finds the pair of image and point cloud data with the closest timestamps and pairs them together. If the time difference is still large, interpolation or prediction methods may be used to compensate for the effects of motion, ensuring that objects in the image and objects in the point cloud are strictly corresponding in space.

[0090] In some embodiments, the robot includes a lawnmower; the step of outputting obstacle recognition results based on the target obstacle and the depth map information includes: obtaining the obstacle avoidance requirements corresponding to the lawnmower; and outputting the obstacle recognition results corresponding to the lawnmower based on the obstacle avoidance requirements, the target obstacle, and the depth map information.

[0091] The final obstacle information output serves the robot's obstacle avoidance and path planning. Therefore, the output obstacle list needs to be combined with the robot's specific "obstacle avoidance requirements." For example, a lawnmower may only care about obstacles within a certain distance (such as 3 meters), or it may need a greater safety distance for specific types of obstacles (such as pets or children).

[0092] In the output module, the system reads preset obstacle avoidance parameters (such as maximum attention distance and safe radius for different types of obstacles). For all target obstacles obtained after fusion, it checks whether their distance is within the attention range and attaches a threat level or obstacle avoidance priority label calculated based on category and distance. The final output is a filtered and sorted list of obstacle information, which is directly used by the path planning module.

[0093] In some embodiments, if the obstacle recognition is a low-light recognition algorithm, the step of completing obstacle recognition for the robot based on the obstacle recognition algorithm and the dense point cloud data includes: acquiring non-ground point cloud data corresponding to the dense point cloud data; acquiring sampling information corresponding to the non-ground point cloud data; acquiring obstacle information based on the sampling information, and completing obstacle recognition for the robot.

[0094] This paper describes the complete steps for obstacle identification using only LiDAR data in low-light conditions. First, non-ground point cloud data is separated from dense point cloud data (i.e., ground points are removed, as the ground is generally not an obstacle avoidance target). Then, the potentially still large amount of non-ground point cloud data is downsampled to reduce computational load, yielding sampled information. Finally, cluster analysis is performed based on the sampled point cloud, grouping points belonging to the same object into a single class, thereby identifying individual obstacles and acquiring their information.

[0095] After the low-light algorithm thread starts, it directly receives the dense point cloud L. First, it performs ground segmentation and filters out ground points. Then, it performs voxel filtering downsampling. Finally, it runs the Euclidean clustering algorithm on the remaining point cloud. Each cluster is regarded as an obstacle, and its 3D bounding box or centroid position and point cloud size can be used as obstacle information output.

[0096] In some embodiments, obtaining the non-ground point cloud data corresponding to the dense point cloud data includes: dividing the dense point cloud data into multiple local blocks according to a preset spatial resolution; obtaining height statistics information corresponding to the point cloud data in each local block; determining whether each local block is a ground local block based on the height statistics information; separating the point cloud data corresponding to the ground local block from the dense point cloud data to obtain the non-ground point cloud data.

[0097] This embodiment divides the entire point cloud space (such as a horizontal plane centered on the robot) into countless small local patches. Plane fitting and statistical analysis are performed on the points within each patch, and the patch is determined to belong to the ground based on its geometric characteristics (such as whether the normal vector of the fitted plane is close to vertical and whether the height variance of the point is small).

[0098] Divide the point cloud into a fixed-size grid (e.g., 10cm x 10cm) on a horizontal plane (XY plane), with points within each grid cylinder forming a local block. Alternatively, a more complex sector-to-ring region division can be used.

[0099] For each local block, a plane is quickly fitted using an algorithm such as RANSAC. The normal vector n of the plane is calculated, along with the mean µ and standard deviation σ of the perpendicular distances from all points within the local block to the fitted plane.

[0100] Set a threshold. If the angle between n and the sky vector (0,0,1) is very small (e.g., <10°), and σ is also small (indicating that the points are all close to the fitted plane), then the local patch is determined to be the ground. Mark and remove the points in all patches that are determined to be the ground; what remains is the non-ground point cloud data.

[0101] In some embodiments, obtaining the height statistics information corresponding to the point cloud data within each local block includes: calculating the plane normal vector of the point cloud data corresponding to the local block; and obtaining the height statistics information of the point cloud data of each local block relative to the fitted plane corresponding to the plane normal vector.

[0102] This embodiment refines the method for calculating height statistics in the embodiments. The core is to obtain a reference plane that can represent the local terrain of a local block as the fitting plane. The normal vector of this plane is calculated, and the directed vertical distance from each point to this fitting plane is used as the "height", and then the statistical characteristics (mean, variance, etc.) of these heights are calculated.

[0103] After fitting the plane using the RANSAC algorithm, the normal vector (A, B, C) of the plane equation Ax + By + Cz + D = 0 can be directly obtained. For any point P(x0, y0, z0) in the patch, its distance to the plane is d = |Ax0 + By0 + Cz0 + D| / (A^2 + B^2 + C^2). 0.5 The set of distances d from all points, whose mean reflects the average height of the patch relative to the fitted plane, and whose standard deviation σ reflects the flatness of points within a local patch. A small σ is an important characteristic of the ground.

[0104] In some embodiments, obtaining the sampling information corresponding to the non-ground point cloud data includes: sampling the non-ground point cloud data based on voxel filtering to obtain the sampling information.

[0105] Even after ground segmentation, the remaining non-ground point cloud may still contain tens of thousands of points, making direct clustering computation computationally intensive. This embodiment employs voxel filtering for downsampling, significantly reducing the number of points while preserving the overall shape characteristics of the point cloud.

[0106] The three-dimensional space is divided into multiple tiny cubic lattices (voxels), such as a cube with a side length of 3 cm. For all points falling within the same voxel, their centroid (or a random point) is used to represent that voxel. This achieves downsampling while avoiding the problem of uneven point cloud density, improving the speed and stability of subsequent processing.

[0107] In some embodiments, obtaining obstacle information based on the sampling information to complete obstacle recognition for the robot includes: obtaining spatial location information and geometric feature information corresponding to the sampling information; obtaining at least one point cloud cluster in the non-ground point cloud data based on a preset Euclidean distance threshold, spatial location information, and geometric feature information; and generating the obstacle information based on the point cloud cluster.

[0108] This embodiment is the final step in obstacle recognition in low-light scenes, aiming to distinguish point clouds belonging to different physical entities. A clustering algorithm based on Euclidean distance is employed.

[0109] Set a threshold `d_th` (e.g., 0.1 meters) based on the minimum gap between objects in the scene. Points smaller than this distance are considered part of the same object. Create an empty list of clusters and an "unvisited" set of all points. Randomly select a point from the unvisited set as a "seed" to start a new cluster. Search for all points within a `d_th` radius of the seed point, add them to the current cluster, and mark them as visited. Repeat the search process using these newly added points as new seeds until no more points are added. At this point, a cluster (i.e., an obstacle point cloud cluster) has grown. Repeat the above steps until all points have been visited. Each point cloud cluster is assigned a unique ID. The 3D bounding box, minimum bounding sphere, number of points, centroid, etc., of each cluster can be calculated as information about the obstacle (position, size, outline) and output.

[0110] In some embodiments, this solution optimizes and upgrades the obstacle recognition technology based on binocular vision modules in traditional lawnmowers, addressing its limitations. In conventional designs, depth information typically relies on binocular cameras to calculate parallax. However, this method is susceptible to interference under complex lighting conditions such as low light or strong light, leading to increased parallax errors and affecting the accuracy and stability of obstacle distance judgment. To improve environmental adaptability and ranging accuracy, this solution innovatively adopts a multi-sensor fusion architecture of "camera + LiDAR". LiDAR directly provides high-precision ground truth distance information, effectively avoiding the sensitivity of traditional binocular vision to lighting conditions. Simultaneously, the camera retains rich texture and semantic information, providing strong support for obstacle recognition and classification. Through the synergistic fusion of these two technologies, the system not only significantly reduces the impact of the external environment on perception accuracy but also greatly improves the robustness and reliability of obstacle recognition, thus providing a more accurate and stable technical guarantee for the lawnmower's intelligent obstacle avoidance and path planning. However, due to the inherent characteristics of radar, the depth is relatively sparse when projected onto the image; therefore, a method of fusing multiple frame point clouds is proposed to densify the depth.

[0111] Single-frame-rate LiDAR point cloud data is dense at close range on a three-dimensional surface, but becomes sparse at medium to long range. It is also sparse when projected onto a two-dimensional pixel plane, which will cause many problems in two-dimensional processing, such as missing detection of small target objects and inaccurate detection of the shape contour of target objects. Therefore, the point cloud is first fused by superimposing the laser point cloud of the historical frame and the current frame to make the point cloud denser. Specifically, the current frame point cloud data Li and pose information Posei (including position information pi and rotation information qi) are obtained, as well as the point cloud Li-1 of the historical adjacent frame and pose information Posei-1 (including position information pi-1 and rotation information qi-1). Then, the historical frame point cloud is transformed into the coordinate system of the current frame point cloud using formula (1): Li 1' = qi.inverse() qi 1 Li + (pi pi 1); The final point cloud data L = Li 1' + Li; This method can overlay historical point clouds from multiple frames onto the current frame to achieve point cloud densification, reduce missed obstacle detection, and optimize obstacle detection.

[0112] like Figure 2 and Figure 3 As shown, this scheme is divided into normal and low-light scene recognition algorithms; In normal scenarios, a camera captures RGB images, providing rich color and texture information; simultaneously, a LiDAR is used to obtain precise distance information. The combination of these two methods compensates for each other's shortcomings, improving the accuracy of obstacle detection. The specific steps are as follows: (I) Image distortion correction: Using camera calibration parameters, distortion correction is performed on the RGB images captured by the camera to eliminate lens distortion effects, preserve texture details, provide high-quality input for subsequent deep learning tasks, and solidify the accuracy foundation for object detection and semantic segmentation. (II) RGB processing: Segmentation and detection are performed in parallel; Semantic segmentation: Through a semantic segmentation network (such as U-Net), the RGB image is subdivided into different semantic regions such as ground and obstacles, initially distinguishing object categories in the scene and providing region-level clues for obstacle recognition. Object detection: Using an object detection model (such as YOLOv11), potential obstacles are identified in the RGB image, outputting bounding boxes and category predictions, accurately locating areas where obstacles may exist. Furthermore, using traditional computer vision techniques (such as SIFT and HOG), additional features are extracted from RGB images, and these features are combined with the results of object detection and semantic segmentation to form a multimodal feature vector. (III) RGB result fusion: complementary verification of detection and segmentation; the semantic segmentation and object detection results are fused, and multi-condition verification rules are formulated to ensure that obstacles are "detected as required"; 1. Strong confirmation: if the object detection marks the area as an obstacle and the semantic segmentation identifies it as non-ground, it is determined to be a real obstacle; 2. Recall compensation: if only the object detection marks the obstacle but the segmentation identifies it as ground, or if only the segmentation map has an obstacle label but the detection does not identify it, the detection box recall mechanism is triggered, and the existence of an obstacle is determined; for obstacle areas that are only marked by object detection or semantic segmentation but not confirmed by the other party. In addition to triggering the recall mechanism, the multimodal feature vector is used and compared with the features of other known obstacles. The decision tree or random forest model is used to make the final judgment based on these features; 3. Double negation: only when there is no obstacle information in the detection and segmentation results is it determined to be obstacle-free. Through complementary verification, more accurate RGB obstacle location and category information is output. (IV) LiDAR processing: Depth information alignment and projection; using the calibration results of camera and radar intrinsic and extrinsic parameters, the LiDAR point cloud is projected onto the RGB image coordinate system to generate a depth map corresponding to the RGB image, where the pixel value represents the distance information of the corresponding position; at the same time, the camera and LiDAR data are aligned based on the timestamp to ensure the temporal consistency of multi-source data fusion. In addition, point cloud density analysis is added: the number of LiDAR points within a certain window around each pixel is counted in the image space, and areas with low density are marked as "low quality" to reduce weight in subsequent fusion. Depth change consistency detection: for each pixel, the difference between its depth value and that of its neighboring pixels is compared. If the difference is too large (such as a sudden change), it may be noise or an edge, and is marked as an "edge / noise region".Generate a quality map: Each pixel corresponds to a quality value (0~1), representing the reliability of the depth information at that point. The quality map is used as a weighting factor in subsequent fusion. (V) Multi-source fusion and obstacle output; Fusion of obstacle information after RGB end-verification and lidar depth information: For potential obstacles identified by RGB, their precise distance is obtained through the depth map; combined with the obstacle avoidance requirements of the lawnmower, obstacles within the obstacle avoidance range are selected, and their position, shape, distance, and other information are output to provide decision-making basis for the lawnmower's intelligent obstacle avoidance and path planning, achieving efficient and accurate obstacle recognition and processing.

[0113] In low-light scenes, cameras cannot effectively identify environmental information and other valid object information. In this case, only radar point clouds are used for obstacle recognition. (I) Patchwork point cloud ground segmentation; Point cloud block division: The original laser point cloud is divided into multiple local patches according to a certain spatial resolution (such as based on grid division, setting an appropriate grid size to balance computational efficiency and segmentation accuracy). Feature calculation: For the point cloud in each patch, plane fitting is performed (such as using the Random Sample Consensus Algorithm - RANSAC to fit the plane), and the normal vector of the plane and the height statistical features of the point cloud in the patch relative to the fitted plane (such as mean, variance, etc.) are calculated; Ground judgment: Based on the preset feature threshold (determined through a large number of experiments or adaptive learning, such as the angle threshold between the normal vector and the vertical direction, the height variance threshold, etc.), it is determined whether each patch is a ground patch. Point clouds belonging to the ground are marked and separated, and the remaining point clouds are non-ground point clouds containing obstacles and other objects. (II) Point Cloud Sampling: The number of non-terrestrial point clouds after patchwork processing is usually still large. To reduce subsequent computational complexity, improve processing efficiency, and retain key geometric features, voxel filtering is used for point cloud sampling. (III) Point Cloud Clustering: The sampled point clouds are clustered into different point cloud clusters based on similarity in spatial location, geometric features, etc. Each cluster corresponds to a possible object (such as pedestrians, vehicles, obstacles, etc.). Distance Threshold Setting: Determine the Euclidean distance threshold (set according to the approximate size of objects in the scene and point cloud density, such as 0.05-0.1m, to ensure that point clouds of the same object can be clustered into one class) as the basis for judging whether points belong to the same cluster. Seed Point Selection and Growth: Randomly select unclassified points as initial seed points, search for unvisited points around them (Euclidean distance less than the threshold), add these points to the current cluster, and continue searching with the newly added points as new seeds until no points that meet the conditions are added, completing the growth of a cluster. Repeat this process until all points are classified, resulting in multiple independent point cloud clusters. The final point cloud retained represents the valid obstacle information.

[0114] The two solutions described above can intelligently switch according to external lighting conditions, adapt to effective obstacle recognition in different environments, and perform reasonable obstacle avoidance functions.

[0115] Please see Figure 4 As shown, Figure 4 This is a schematic diagram of the obstacle recognition device 200 provided in the embodiments of this application. The obstacle recognition device 200 is used to perform the steps of the obstacle recognition method shown in the above embodiments. The obstacle recognition device 200 can be a single server or a server cluster, or the obstacle recognition device 200 can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0116] like Figure 4 As shown, the obstacle recognition device 200 includes: The data acquisition unit 201 is used to acquire the current frame point cloud data and the historical adjacent frame point cloud data corresponding to the robot, and to acquire dense point cloud data based on the current frame point cloud data and the historical adjacent frame point cloud data.

[0117] The information acquisition unit 202 is used to acquire obstacle information based on the dense point cloud data and complete obstacle recognition for the robot.

[0118] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the obstacle recognition device and its modules described above can be referred to the corresponding processes in the obstacle recognition method embodiments described above, and will not be repeated here.

[0119] The obstacle recognition method described above can be implemented as a computer program, which can, for example... Figure 4 It runs on the device shown.

[0120] Please see Figure 5 , Figure 5 This is a schematic block diagram of the robot provided in an embodiment of this application. The robot includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0121] The storage medium may store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any obstacle recognition method.

[0122] The processor provides computing and control capabilities to support the operation of the entire robot.

[0123] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any obstacle recognition method.

[0124] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. A specific robot may include more or fewer parts than shown in the figure, or combine certain parts, or have different part arrangements.

[0125] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0126] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Obtain the current frame point cloud data and historical adjacent frame point cloud data corresponding to the robot, and obtain dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data.

[0127] Obstacle information is obtained based on the dense point cloud data, and obstacle recognition is completed for the robot.

[0128] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method provided in any embodiment of this application.

[0129] The computer-readable storage medium can be the internal storage unit of the robot described in the foregoing embodiments, such as the robot's hard drive or memory. Alternatively, the computer-readable storage medium can be an external storage device for the robot, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An obstacle recognition method, characterized in that, Applied to robots; the method includes: Obtain the current frame point cloud data and historical adjacent frame point cloud data corresponding to the robot, and obtain dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data. Obstacle information is obtained based on the dense point cloud data, and obstacle recognition is completed for the robot.

2. The method according to claim 1, characterized in that, The step of obtaining dense point cloud data based on the current frame point cloud data and historical adjacent frame point cloud data includes: Obtain the current frame pose information and historical adjacent frame pose information of the robot; Based on the current frame pose information and the pose information of historical adjacent frames, the point cloud data of historical adjacent frames is transformed into the point cloud coordinate system of the current frame. The densed point cloud data is obtained by combining the converted historical adjacent frame point cloud data with the current frame point cloud data.

3. The method according to claim 2, characterized in that, The current frame pose information includes current position information and current rotation information, and the historical adjacent frame pose information includes historical position information and historical rotation information; the step of converting the historical adjacent frame point cloud data to the current frame point cloud coordinate system based on the current frame pose information and the historical adjacent frame pose information includes: The historical adjacent frame point cloud data is converted according to the current rotation information and the historical rotation information, so as to adjust the rotation attitude corresponding to the historical adjacent frame point cloud data to the rotation attitude of the current frame. Obtain the position difference between the historical location information and the current location information; Based on the position difference and the transformed historical adjacent frame point cloud data, the historical adjacent frame point cloud data is transformed into the current frame point cloud coordinate system.

4. The method according to claim 3, characterized in that, The step of converting the historical adjacent frame point cloud data based on the current rotation information and the historical rotation information includes: Perform an inverse transformation on the current rotation information; The historical adjacent frame point cloud data are transformed based on the current rotation information after inverse transformation and the historical rotation information.

5. The method according to claim 1, characterized in that, The step of obtaining obstacle information based on the dense point cloud data includes: Obtain the external lighting conditions corresponding to the robot, and determine the corresponding obstacle recognition algorithm based on the external lighting conditions; Obstacle information is obtained based on the obstacle recognition algorithm and the dense point cloud data.

6. The method according to claim 5, characterized in that, The obstacle recognition algorithm includes at least a normal environment recognition algorithm and a low-light recognition algorithm; the step of determining the corresponding obstacle recognition algorithm based on the external lighting conditions includes: Acquire real-time environmental images captured by the image acquisition device mounted on the robot; The real-time environment image is acquired and grayscale processed, and the average brightness value corresponding to the grayscale real-time environment image is calculated. The average brightness value is compared with a preset brightness threshold. If the average brightness value is greater than or equal to the brightness threshold, the obstacle recognition algorithm is determined to be the normal environment recognition algorithm; if the average brightness value is less than the brightness threshold, the obstacle recognition algorithm is determined to be the low light recognition algorithm.

7. The method according to claim 5, characterized in that, If the obstacle recognition algorithm is a normal environment recognition algorithm, the step of completing obstacle recognition for the robot based on the obstacle recognition algorithm and the dense point cloud data includes: The robot acquires the image to be identified. Detect at least one target obstacle in the image to be identified; The dense point cloud data is projected onto the image coordinate system corresponding to the image to be identified in order to obtain depth map information; The obstacle identification result is output based on the target obstacle and the depth map information; the obstacle identification result includes obstacle information corresponding to each target obstacle, and the obstacle information includes at least one or more of the following: location information, shape information and distance information.

8. The method according to claim 7, characterized in that, Before detecting at least one potential obstacle in the image to be identified, the method further includes: The distortion correction is performed on the image to be identified according to the preset camera calibration parameters.

9. The method according to claim 7, characterized in that, The step of detecting at least one target obstacle in the image to be identified includes: The image to be identified is semantically segmented to obtain multiple semantic regions; Target detection is performed on the image to be identified to obtain at least one potential obstacle and the obstacle information corresponding to the potential obstacle; Determine the target semantic region corresponding to each potential obstacle within the plurality of semantic regions; Whether a potential obstacle is the target obstacle is determined based on the obstacle information and target semantic region corresponding to each potential obstacle.

10. The method according to claim 9, characterized in that, The step of determining whether a potential obstacle is the target obstacle based on the obstacle information and target semantic region corresponding to each potential obstacle includes: Obtain the semantic tag information corresponding to each of the semantic regions; If, based on the semantic tag information, it is determined that the target semantic region corresponding to the potential obstacle is not ground, then the potential obstacle is determined to be the target obstacle; and / or, If the target semantic region corresponding to the potential obstacle is determined to be the ground based on the semantic tag information, the obstacle information corresponding to the potential obstacle and the semantic tag information corresponding to the target semantic region are input into a preset obstacle recognition model to determine whether the potential obstacle is the target obstacle; and / or, If, based on the semantic tag information, it is determined that any semantic region contains an obstacle tag but has no corresponding potential obstacle, the semantic tag information corresponding to the semantic region is input into a preset obstacle recognition model to determine whether the potential obstacle is the target obstacle.

11. The method according to claim 7, characterized in that, The robot is equipped with an image acquisition device and a lidar. The lidar is used to acquire the point cloud data of the current frame, and the image acquisition device is used to acquire the image to be identified. The step of projecting the dense point cloud data onto the image coordinate system corresponding to the image to be identified includes: Obtain the internal and external parameter calibration information corresponding to the image acquisition device and the lidar; Based on the intrinsic and extrinsic parameter calibration information, the dense point cloud data is projected onto the image coordinate system corresponding to the image to be identified in order to obtain the depth map information.

12. The method according to claim 7, characterized in that, Before outputting the obstacle recognition result based on the target obstacle and the depth map information, the method further includes: In the image space corresponding to the image to be identified, the point cloud density information corresponding to each pixel is obtained according to the dense point cloud data; Quality information is generated based on the point cloud density information, and the obstacle recognition result is output based on the target obstacle, the depth map information, and the quality information.

13. The method according to claim 7, characterized in that, Before projecting the dense point cloud data onto the image coordinate system corresponding to the image to be identified, the method further includes: Obtain the first timestamp information corresponding to the dense point cloud data and the second timestamp information corresponding to the image to be identified; The dense point cloud data and the image to be identified are time-stamped according to the first timestamp information and the second timestamp information.

14. The method according to claim 7, characterized in that, The robot includes a lawnmower; the step of outputting obstacle recognition results based on the target obstacle and the depth map information includes: Obtain the obstacle avoidance requirements corresponding to the lawnmower; Based on the obstacle avoidance requirements, the target obstacle, and the depth map information, the obstacle identification result corresponding to the lawnmower is output.

15. The method according to claim 5, characterized in that, If the obstacle recognition algorithm is a low-light recognition algorithm, the step of completing obstacle recognition for the robot based on the obstacle recognition algorithm and the dense point cloud data includes: Obtain the non-ground point cloud data corresponding to the dense point cloud data; Obtain the sampling information corresponding to the non-ground point cloud data; Obstacle information is obtained based on the sampling information, and obstacle recognition of the robot is completed.

16. The method according to claim 15, characterized in that, The step of obtaining the non-ground point cloud data corresponding to the dense point cloud data includes: The dense point cloud data is divided into multiple local blocks according to a preset spatial resolution; Obtain the height statistics of the point cloud data within each local block; Determine whether each local block is a ground local block based on the height statistics; The point cloud data corresponding to the local ground block is separated from the dense point cloud data to obtain the non-ground point cloud data.

17. The method according to claim 16, characterized in that, The process of obtaining the height statistics information corresponding to the point cloud data within each local block includes: Calculate the plane normal vector of the point cloud data corresponding to the local block; Obtain the height statistics of the fitted plane corresponding to the point cloud data of each local block relative to the plane normal vector.

18. The method according to claim 15, characterized in that, The step of obtaining the sampling information corresponding to the non-ground point cloud data includes: The non-ground point cloud data is sampled based on voxel filtering to obtain the sampling information.

19. The method according to claim 15, characterized in that, The step of obtaining obstacle information based on the sampling information and completing obstacle recognition for the robot includes: Obtain the spatial location information and geometric feature information corresponding to the sampling information; At least one point cloud cluster is obtained from the non-ground point cloud data based on a preset Euclidean distance threshold, spatial location information, and geometric feature information. The obstacle information is generated based on the point cloud cluster.

20. A robot, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 19.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer-readable instructions of which, when executed by the processor, cause one or more processors to perform the steps of the method as described in any one of claims 1 to 19.

Images