Wall surface identification method and device and storage medium

Abstract

The invention discloses a wall surface identification method and device and a storage medium, and relates to the technical field of sensor positioning, and the method comprises the steps: carrying out the fitting processing of a laser radar point cloud collected by a laser point cloud radar, and obtaining a plurality of fitting line segments; screening a target line segment which is greater than a first length threshold and has the longest length from the plurality of fitting line segments; projecting target point clouds associated with the position of the target line segment in wall point clouds and obstacle point clouds collected by the depth camera on the target line segment, and determining at least one target wall surface line segment; and if the length of the target wall surface line segment is greater than a line segment threshold value, outputting the target wall surface line segment under a robot coordinate system to be used by a welting program. According to the invention, after laser point cloud filtering fitting, depth camera wall surface obstacle point cloud projection calibration is fused, the technical problem that the wall surface is difficult to effectively identify is solved, and the wall attaching precision and obstacle avoidance reliability of the cleaning robot in a complex scene are improved.

Description

Technical Field

[0001] The present application relates to the field of sensor positioning technology, and in particular to a wall recognition method, device, and storage medium. Background Art

[0002] As an important branch of the field of intelligent service robots, cleaning robots have gradually achieved adaptability to complex scenarios in recent years through the combination of multimodal perception and autonomous navigation technology. However, cleaning robots need to operate close to the wall during operation. The relevant technology is limited by the size of the robot body, and there are blind spots in the field of vision when operating at close range, making it difficult to effectively identify the wall.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a wall recognition method, device and storage medium, aiming to solve the technical problem in related technologies that due to the limitation of the robot body size, there are blind spots in the field of vision when working at close range, which makes it difficult to effectively identify the wall.

[0005] To achieve the above objectives, the present application proposes a wall recognition method, which includes:

[0006] Fitting the laser radar point cloud collected by the laser point cloud radar to obtain multiple fitting line segments;

[0007] Filtering a target line segment with a length greater than a first length threshold and the longest length among the plurality of fitted line segments;

[0008] Projecting the target point cloud associated with the position of the target line segment in the wall point cloud and the obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment;

[0009] If the length of the target wall line segment is greater than the line segment threshold, the target wall line segment in the robot coordinate system is output for use by the edge-fitting program.

[0010] In one embodiment, the laser radar data of the current frame is converted into the robot coordinate system to obtain coordinate point cloud data;

[0011] The coordinate point cloud data is projected onto the XY plane to obtain the laser radar point cloud.

[0012] In one embodiment, the space where the laser radar point cloud is located is evenly divided to determine a plurality of identical grids;

[0013] Aggregating the point cloud portion within each of the grids to obtain a filtered lidar point cloud;

[0014] Based on the filtered lidar point cloud, straight line fitting and cluster decomposition are performed to determine a plurality of fitting line segments.

[0015] In one embodiment, based on a comparison between the plurality of fitted line segments and the first length threshold, a plurality of fitted line segments that are greater than the first length threshold are used as standard line segments;

[0016] The lengths of the plurality of standard line segments are compared, and the standard line segment with the longest length is selected as the target line segment.

[0017] In one embodiment, filtering is performed on the depth camera data projected onto the XY plane to determine standard point cloud data corresponding to the depth camera data;

[0018] Taking the center point of the depth camera as the origin, divide the angle grid and select the standard point cloud data farthest from the origin in each grid as the calculation point;

[0019] A fitting clustering process is performed based on the calculated points to generate initial line segments.

[0020] In one embodiment, based on a second length threshold, the initial line segment is compared with the second length threshold;

[0021] If the initial line segment is greater than the second length threshold, marking the initial line segment as a wall on the map;

[0022] Based on the initial line segment marked as a wall, the obstacle point cloud within a preset first distance threshold from one side of the robot is marked on the map as an obstacle.

[0023] In one embodiment, point cloud data whose distance from the initial line segment is less than a second distance threshold is used as the wall point cloud, and point cloud data whose distance from the initial line segment is greater than the second distance threshold is used as the obstacle point cloud;

[0024] Projecting the wall point cloud onto the target line segment, clustering the projected wall point cloud to generate a target projected line segment;

[0025] The obstacle point cloud is projected onto the target projection line segment to generate at least one target wall line segment.

[0026] In one embodiment, a prediction result is generated based on the edge distance between the target wall segment and one side of the robot and a preset target edge distance;

[0027] If the prediction result shows that the edge distance is less than the preset target edge distance, generating an adjustment strategy corresponding to the prediction result;

[0028] According to the adjustment strategy, the path of the robot is adjusted to keep the edge distance equal to the preset target edge distance.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes a wall recognition device, which includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor, and the computer program is configured to implement the steps of the wall recognition method described above.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the wall recognition method as described above are implemented.

[0031] The present application provides a method for identifying a wall, comprising fitting a laser radar point cloud collected by the laser point cloud radar to obtain a plurality of fitting line segments; screening a target line segment with a length greater than a first length threshold and the longest length among the plurality of fitting line segments; projecting a target point cloud associated with the position of the target line segment in the wall point cloud and obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment; if the length of the target wall line segment is greater than the line segment threshold, outputting the target wall line segment in the robot coordinate system for use by the edge fitting program. The present application uses a laser radar and a depth camera to respectively identify the wall and the obstacle, and then obtains the wall recognition result by combining the data, with high recognition accuracy.

[0032] To summarize, after laser point cloud filtering and fitting, this application selects the longest target line segment according to the length threshold, integrates the depth camera wall obstacle point cloud projection calibration, and outputs the valid line segment for edge control, so that the edge control program can obtain the wall posture with centimeter-level accuracy in real time, overcoming the technical problems in related technologies that are limited by the size of the robot body and have blind spots in the field of view when working at close range, making it difficult to effectively identify the wall. This improves the cleaning robot's wall-sticking accuracy and obstacle avoidance reliability in complex scenes, while reducing the system computing load. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 This is a flow chart of the first embodiment of the wall recognition method of the present application;

[0036] Figure 2 This is a flow chart of the third embodiment of the wall recognition method of the present application;

[0037] Figure 3 This is a flow chart of a fifth embodiment of the wall recognition method of the present application;

[0038] Figure 4 This is a flow chart of a sixth embodiment of the wall recognition method of the present application;

[0039] Figure 5 This is a flowchart of the wall recognition method of this application;

[0040] Figure 6 Keep a schematic diagram of the area for the depth camera point cloud;

[0041] Figure 7 Schematic diagram of the establishment of the angle grid and the selection of calculation points;

[0042] Figure 8 This is a schematic diagram of LiDAR data processing;

[0043] Figure 9 Schematic diagram of map point selection and projection;

[0044] Figure 10 is a schematic diagram of the target wall segment;

[0045] Figure 11 This is a schematic diagram of the structure of the wall identification device of this application.

[0046] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0048] The related technology is limited by the size of the robot body, and there are blind spots in the field of vision when working at close range, making it difficult to effectively identify the wall.

[0049] The present application provides a solution: first, the laser radar point cloud collected by the laser point cloud radar is fitted and processed to obtain multiple fitting line segments, and then the target line segment with a length greater than a first length threshold and the longest length among the multiple fitting line segments is screened, and then the target point cloud associated with the position of the target line segment in the wall point cloud and obstacle point cloud collected by the depth camera is projected onto the target line segment to determine at least one target wall line segment, and finally, if the length of the target wall line segment is greater than the line segment threshold, the target wall line segment in the robot coordinate system is output for use by the edge fitting program.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, a wall recognition device, etc. The following uses a wall recognition device as an example to illustrate this embodiment and the following embodiments.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The present application embodiment provides a method for identifying a wall, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the wall recognition method of the present application.

[0053] In this embodiment, the wall recognition method includes steps S10 to S40:

[0054] Step S10: Fitting the laser radar point cloud collected by the laser point cloud radar to obtain multiple fitting line segments.

[0055] In this embodiment, the lidar point cloud refers to the point cloud data extracted from the XY plane by converting the lidar data of the current frame into the robot coordinate system for processing and projecting it onto the XY plane. Fitting processing refers to clustering discrete points into straight line equations using a random sampling consensus algorithm or a least squares method. The first length threshold refers to the minimum length value for determining a valid wall segment for the lidar point cloud. The target segment refers to the longest valid straight line after length screening. The wall point cloud refers to the set of wall structure points extracted by the depth camera through segmentation and aggregation. The obstacle point cloud refers to the point cloud data of obstacles on the wall. The segment threshold is the minimum valid length of the final output segment.

[0056] As an optional implementation method, the lidar point cloud on the right side of the robot is partially downsampled to a voxel grid, the lidar point cloud is evenly divided into space, multiple identical grids are determined, the point cloud parts within the grids are aggregated, and the point cloud part farthest from the origin in each angle grid is taken as the calculation point. Finally, the lidar point cloud is fitted using a random sampling consistency algorithm to generate multiple fitting line segments.

[0057] As an optional implementation method of the target wall segment, in the scenario of determining the length of the target wall segment, if the length of the target wall segment is less than the segment threshold, a result indicating that the wall search failed is output.

[0058] Step S20 , screening a target line segment with a length greater than a first length threshold and the longest among the plurality of fitted line segments.

[0059] In this embodiment, the first length threshold refers to a custom-defined minimum valid segment length. The fitted segment refers to the equation of a line generated from the original point cloud using an algorithm. The target segment is the longest segment that meets the length threshold and represents the candidate wall reference.

[0060] As an optional implementation, the first length threshold is compared with multiple fitted line segments, and the fitted line segments greater than the first length threshold are screened out. Then, the lengths of these fitted line segments are compared, and the fitted line segment with the longest length is screened out as the target line segment.

[0061] As an optional sensor implementation method, this solution has low requirements for sensor layout. The current conventional depth camera and 2D lidar combination layout of commercial cleaning robots can basically meet the requirements of this algorithm.

[0062] Step S30 : Projecting the target point cloud associated with the position of the target line segment in the wall point cloud and the obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment.

[0063] In this embodiment, the wall point cloud refers to the 3D point set of the wall surface extracted by the depth camera through plane segmentation. The obstacle point cloud refers to the 3D point set of non-wall dynamic objects identified by the depth camera. The target line segment refers to the longest valid line segment fitted by the lidar. The target point cloud refers to the point cloud data associated with the target line segment position in the wall point cloud and obstacle point cloud. Projection refers to projecting the wall point cloud and obstacle point cloud onto the target line segment. The target wall line segment refers to the continuous valid segment occupied by the projected wall point cloud.

[0064] As an optional implementation method, the wall point cloud and obstacle point cloud obtained by the depth camera are projected onto the plane where the target line segment is located, and merged with the wall point cloud along the direction of the target line segment. The target point cloud associated with the corresponding position of the target line segment is disconnected based on the projection position of the obstacle point cloud to obtain the target wall line segment.

[0065] As an optional implementation method for determining the target wall line segment, in a scenario where the wall point cloud and the obstacle point cloud are projected onto the target line segment, if there is no obstacle point cloud at the associated position of the target line segment, a target wall line segment is determined.

[0066] As an optional implementation method for determining the target wall line segment, in a scenario where the wall point cloud and the obstacle point cloud are projected onto the target line segment, if an obstacle point cloud exists at the associated position of the target line segment, the obstacle point cloud is projected onto the associated position of the corresponding target line segment, and the corresponding position of the target line segment where the obstacle point cloud exists is disconnected to determine multiple target wall line segments.

[0067] Step S40: If the length of the target wall line segment is greater than the line segment threshold, the target wall line segment in the robot coordinate system is output for use by the edge-adapting program.

[0068] In this embodiment, the line segment threshold refers to the minimum length required to determine wall validity and to filter out fragmented data. The target wall segment refers to the spatial geometric representation of the wall generated through projection processing. The robot coordinate system refers to the local rectangular coordinate system with the robot center as the origin. The edge tracking program refers to a closed-loop control algorithm that generates trajectories in real time based on the wall position.

[0069] As an optional implementation, the target wall line segment whose length exceeds the line segment threshold is output, and the three-dimensional coordinates and direction vector of the starting point of the target wall line segment in the robot local coordinate system are transmitted to the edge program.

[0070] As an optional implementation method of determining the target wall line segment, if the length of the target wall line segment is determined to be less than the line segment threshold, a result indicating that the wall detection has failed is obtained and output.

[0071] For example, voxel grid filtering (resolution 0.05m) and statistical outlier filtering (standard deviation threshold 1.5) are performed in real time on the original point cloud collected by the laser radar on the right side of the robot, and multiple line segments are fitted using the RANSAC algorithm; the longest line segment exceeding the threshold is selected as the target reference axis based on the preset length threshold (0.6m); the wall point cloud and obstacle point cloud extracted by the depth camera through plane segmentation are simultaneously projected to the two-dimensional plane where the target line segment is located through coordinate system transformation, and an occupancy histogram is constructed along the direction of the target line segment. The segment occupied by the continuous wall is extracted in combination with the obstacle point cloud to generate the target wall line segment. If the length of the line segment is ≥ the preset effective threshold (0.4m), its endpoint coordinates are converted to the robot coordinate system through the pose transformation matrix, and the structured line segment data with direction vector is output to the real-time control module of the edge controller.

[0072] After the laser point cloud is filtered and fitted, the longest target line segment is selected according to the length threshold, and the depth camera wall obstacle point cloud projection calibration is integrated to output the valid line segment for edge control. The edge control program can obtain the wall posture with centimeter-level accuracy in real time, overcoming the technical problem in related technologies that are limited by the robot body size and have blind spots in the field of view when working at close range, making it difficult to effectively identify the wall. This improves the cleaning robot's wall-sticking accuracy and obstacle avoidance reliability in complex scenes, while reducing the system computing load.

[0073] Based on any of the above embodiments, in the second embodiment of the present application, before step S10, steps A11 to A12 are further included:

[0074] Step A11: convert the laser radar data of the current frame into the robot coordinate system to obtain coordinate point cloud data.

[0075] In this embodiment, the laser radar data of the current frame refers to the set of original point cloud data acquired in the latest scanning cycle.

[0076] As an optional implementation, the laser radar data collected in the current frame is converted into a coordinate system, a Cartesian coordinate system is constructed with the robot as the coordinate origin, and the laser radar data is converted into the Cartesian coordinate system to generate coordinate point cloud data.

[0077] Step A12: Project the coordinate point cloud data onto the XY plane to obtain the lidar point cloud.

[0078] In this embodiment, the XY plane refers to a horizontal two-dimensional plane with the robot's forward direction as the X-axis and the left side as the Y-axis. Projection refers to the vertical mapping of two-dimensional points onto the XY plane. The initial point cloud data refers to the two-dimensional projection point set formed after projection.

[0079] As an optional implementation, the coordinate point cloud data is projected, and then the invalid outliers are filtered using a bounding box constraint, and the remaining point cloud projection is filtered. Finally, based on the point cloud projection, initial point cloud data containing only two-dimensional coordinates is generated.

[0080] Exemplarily, the pose transformation matrix from the laser radar to the robot base is obtained in real time, the laser radar data of the original polar coordinates of the current frame is converted into point cloud data in the robot coordinate system, the point cloud is vertically projected onto the plane of the robot's forward direction by forcing the Z coordinate to zero, an initial two-dimensional point set is generated, outliers exceeding plus or minus five meters in the X direction or plus or minus three meters in the Y direction are filtered out, the XY plane is divided into uniform spatial units based on a preset 0.2-meter grid resolution, the initial point cloud is assigned to the corresponding grid through a hash mapping table, a density clustering algorithm is performed on the point cloud in each grid to merge neighboring points with a spacing less than 0.1 meter, and clusters with a boundary spacing less than 0.15 meters are fused across grids, the valid area is screened based on geometric feature constraints, the angle between the main direction of the area and the transverse axis of the robot must be between fifty-five and one hundred and twenty-five degrees, the Y coordinate of the center of mass of the area must be greater than 0.1 meter, and the minimum enclosing rectangle covers an area greater than 0.15 square meters on the positive Y semi-axis, and finally the coordinate point cloud data after boundary smoothing is output.

[0081] Due to the combination of dynamic coordinate transformation and voxel filtering, the structured right point cloud is generated in real time, which overcomes the problem of inaccurate wall feature extraction and improves the accuracy of the wall recognition method.

[0082] Based on any of the above embodiments, in the third embodiment of the present application, refer to Figure 2 , Figure 2 This is a flow chart of the third embodiment of the wall recognition method of the present application. Step S10 includes steps B11 to B13:

[0083] Step B11: evenly divide the space where the laser radar point cloud is located to determine multiple identical grids.

[0084] In this embodiment, spatial uniform division refers to dividing the point cloud distribution area into equal-sized grid cells at a fixed resolution. Identical grids refer to all grid cells having completely consistent geometric dimensions and coordinate system orientations.

[0085] As an optional implementation, the spatial region where the lidar point cloud is located is evenly divided to obtain multiple grids of the same size.

[0086] As an implementation method of voxel filtering, the three-dimensional space is evenly divided into multiple tiny cubic units, and all point cloud data within each voxel are simplified.

[0087] Step B12: Aggregate the point cloud parts within each grid to obtain a filtered lidar point cloud.

[0088] In this embodiment, the point cloud within a grid refers to the raw LiDAR point set contained in each cubic unit after uniform spatial division. Aggregation involves performing noise filtering, neighbor point fusion, and spatial continuity reconstruction on the point clouds within the same grid. The filtered LiDAR point cloud refers to the structured point cloud that has undergone voxel filtering and geometric optimization.

[0089] As an optional implementation, the point cloud parts in each grid are aggregated, the point farthest from the origin in each grid is selected as the calculation point, and the calculation points are aggregated into a set to obtain the filtered lidar part.

[0090] Step B13: performing straight line fitting and cluster decomposition based on the filtered lidar point cloud to determine a plurality of fitting line segments.

[0091] In this embodiment, cluster decomposition refers to segmenting a point cloud into multiple independent line segment clusters based on differences in geometric features.

[0092] As an optional implementation method, first, based on the region growing clustering algorithm, the point cloud is decomposed into multiple independent plane clusters with the point spacing less than the preset point spacing and the normal vector angle less than the preset point angle as the merging conditions. Then, the random sampling consistency algorithm is applied to each plane cluster to iteratively fit the straight line model. Two points are randomly selected each time to generate the straight line equation. The distance from the point in the cluster to the straight line is calculated and the number of inner points is counted. The optimization model with the inner point rate exceeding the preset probability and the line segment length greater than 0.3 meters is selected. Then, the angle and endpoint spacing of adjacent fitted segments are detected. If the angle is less than the preset segment angle and the endpoint spacing is less than the preset endpoint spacing, the segment is merged, and finally multiple fitted segments with directional vectors are output.

[0093] Exemplarily, the laser radar point cloud space on the right side of the robot is divided into three-dimensional uniform segments, and a hash index table is established to assign the point cloud to the corresponding grid unit. Three-level aggregation processing is performed on each grid. First, discrete noise points are eliminated by statistical filtering with a standard deviation of 0.08 meters. Then, adjacent points are merged through Euclidean clustering with a radius of 0.1 meter. Finally, a continuous surface is generated by cross-cell fusion with 26 neighborhood grid clusters (spacing threshold 0.15 meters). Based on the aggregated point cloud, independent plane clusters are segmented using regional growing clustering (point distance threshold 0.1 meter / normal angle difference 5 degrees). Random sampling consistency algorithm straight line fitting is iteratively performed within each cluster (inlier distance threshold 0.02 meters, inlier rate 85%). If the angle between adjacent line segments is less than 20 degrees and the end point distance is less than 0.1 meter, merging is performed and multiple fitted line segments are output.

[0094] By performing voxel filtering, straight line fitting, and cluster decomposition on the lidar point cloud, the problems of feature fracture caused by the spatial discreteness of the point cloud and line segment fitting distortion caused by noise interference in complex environments are solved, thereby improving the accuracy of the wall recognition method.

[0095] Based on any of the above embodiments, in the fourth embodiment of the present application, step S20 includes steps C11 to C12:

[0096] Step C11 : Based on the comparison between the plurality of fitted line segments and the first length threshold, the plurality of fitted line segments having lengths greater than the first length threshold are used as standard line segments.

[0097] In this embodiment, a standard line segment refers to a valid line segment whose length exceeds a threshold and meets the geometric continuity requirement.

[0098] As an optional implementation, the endpoint coordinates of all fitted line segments are traversed, the Euclidean distance is calculated as the segment length corresponding to each fitted line segment using the distance formula, the segment length corresponding to each fitted line segment is compared with the first length threshold one by one, and all fitted line segments with segment lengths greater than the threshold are screened as standard line segments.

[0099] Step C12: Compare the lengths of the plurality of standard line segments and select the standard line segment with the longest length as the target line segment.

[0100] In this embodiment, length comparison refers to calculating the Euclidean distance of each line segment and performing a descending numerical comparison.

[0101] As an optional implementation, the standard line segments are sorted according to the line segment length. After the sorting is completed, the line segment length is reviewed to determine that the standard line segments are arranged in sequence according to the corresponding line segment length, and the standard line segment with the longest line segment length is selected as the target line segment.

[0102] As an optional implementation method for determining the target line segment, the endpoint coordinates of all standard line segments are traversed, the length value of each line segment is calculated in real time and stored in a dynamic array, and the line segments are sorted in descending order from large to small according to the length value through a quick sort algorithm. The first line segment in the sorting result is selected as the target line segment, and the length difference between the longest line segment and the second longest line segment is detected at the same time. If the difference does not meet the requirements, the geometric feature review is started, and the cosine value of the angle between the direction vectors of the two line segments and the endpoint spacing are compared. If the conditions are met, the line segments are merged and the longest line segment is extracted again. Finally, the start and end coordinates of the target line segment, the line segment length, the direction vector and the unique ID are output.

[0103] For example, the endpoint coordinates of all fitted line segments are traversed in real time to calculate their lengths, and line segments with length values ​​greater than a first length threshold (0.6 meters) are screened and stored as standard line segments in a dynamic array. The line segments in the array are sorted in descending order of length, and the direction angles between adjacent line segments (cosine values ​​greater than 0.975 correspond to an internal angle difference of twelve degrees) and the endpoint spacing (less than 0.15 meters) are detected. If the conditions are met, they are merged into a composite line segment; the longest line segment after sorting is selected as the target line segment, and if the length difference between the longest and second longest line segments is less than 0.1 meter and the geometric features are discontinuous, the original longest line segment is retained; and finally, the precise start / end point coordinates of the target line segment (with an error of less than 1.5 centimeters), three-dimensional direction vector and unique identification code are output.

[0104] By presetting the length threshold, the target line segments that meet the requirements can be quickly screened out, overcoming the problems of edge navigation jitter caused by interference from short line segments and inaccurate reference axis selection in scenarios with multiple walls, thereby improving the stability of the wall recognition method.

[0105] Based on any of the above embodiments, in the fifth embodiment of the present application, refer to Figure 3 , Figure 3 This is a flowchart of the fifth embodiment of the wall recognition method of the present application. Before step C30, steps D11 to D13 are also included:

[0106] Step D11 : performing filtering processing on the depth camera data projected onto the XY plane to determine standard point cloud data corresponding to the depth camera data.

[0107] In this embodiment, the projection of depth camera data onto the XY plane refers to the two-dimensional point set formed by forcing the Z=0 coordinate transformation of the three-dimensional point cloud captured by the depth camera. Filtering refers to the cleaning process of removing noise, outliers, and invalid data through algorithms. Standard point cloud data refers to a reliable point cloud with spatial continuity after geometric correction and density optimization.

[0108] As an optional implementation, the original point cloud projected by the depth camera on the XY plane of the robot is filtered. First, radius filtering is used to remove sparse noise with less than a preset number of neighborhood points within a preset radius. Then, statistical outlier filtering is used to eliminate discrete points whose average distance from neighboring points exceeds a preset multiple of the standard deviation. Then, Euclidean clustering is applied to retain continuous areas with point spacing less than the preset point spacing. Finally, voxel grid downsampling is performed along the main direction to achieve point cloud homogenization, and the output smooth point cloud set is used as standard point cloud data.

[0109] As an optional implementation method for obtaining depth camera data, in a scenario where the robot is not driving straight forward along the edge, the depth camera detects whether there is a wall directly in front of the robot, determines the wall and its corresponding obstacles by collecting point cloud data formed directly in front, and marks the determined data on the map.

[0110] As an optional implementation method for obtaining depth camera data, in a scenario where the robot is driving close to the edge of the road, the depth camera projects a point cloud onto the wall diagonally in front of the robot, and collects point cloud data formed on the wall diagonally in front of the robot to determine the wall diagonally in front and its corresponding obstacles.

[0111] In step D12, the center point of the depth camera is used as the origin to divide the grid into angle grids, and the standard point cloud data farthest from the origin in each grid is selected as the calculation point.

[0112] In this embodiment, the center point of the depth camera refers to the origin of the coordinate system of the optical center of the depth camera (X=0, Y=0, Z=0). The angle grid refers to the sector area of ​​360° horizontal space divided into fixed angle intervals with the origin as the center. The standard point cloud data refers to the reliable point cloud after filtering. The calculation point farthest from the origin refers to the spatial point farthest from the origin in each angle grid.

[0113] As an optional implementation, based on the origin of the depth camera as the center of the coordinate system, the horizontal space is divided into multiple sector-shaped areas, the angle value relative to the origin is calculated for each point in the standard point cloud data and assigned to the corresponding sector-shaped area, and the valid point farthest from the origin in a single sector-shaped area is selected as the calculation point.

[0114] As an optional implementation method for determining the calculation points, if there is no valid point in the area, a null value is marked, and finally the farthest point sets of multiple fan-shaped areas are output, and a continuous space boundary contour is formed based on the farthest point sets.

[0115] Step D13: performing fitting clustering processing based on the calculated points to generate initial line segments.

[0116] In this embodiment, the fitting clustering process refers to grouping the spatially distributed calculation points according to geometric continuity and distance constraints and fitting them into line segments. The initial line segments refer to a set of straight line segments with endpoint coordinates generated by fitting.

[0117] As an optional implementation method, the calculation points of each sector area are traversed, and the angle interval of adjacent points is first detected. If the angle difference between consecutive points is less than the preset angle difference and the distance difference is the preset distance difference, they are marked as the same cluster. Then, a random sampling consistency algorithm is applied to iteratively fit the straight line model within a single cluster. Two points are randomly selected each time to generate a straight line equation. If the proportion of inliers whose straight line distance is less than the preset inlier distance threshold exceeds 85%, the model is retained. Finally, a geometric check is performed on the fitted line segments, and broken line segments whose angles between adjacent line segments are less than the preset angle and whose endpoint spacing is less than the preset spacing are merged, and the initial line segments with start and end point coordinates and direction vectors are output.

[0118] For example, serial filtering is performed on the original point cloud projected by the depth camera to the XY plane: first, isolated noise points are removed by radius filtering (neighborhood radius 20cm / minimum number of points 15), then outliers are removed by statistical filtering (standard deviation multiple 3.0), and finally continuous areas are extracted by Euclidean clustering (point distance threshold 10cm / minimum number of cluster points 30) to generate standard point cloud data with an error of <1cm; the 360° space is divided into 72 5° sector grids with the origin of the depth camera as the center. The standard point cloud data is traversed to calculate the angle of each point and assign it to the corresponding grid. The point in each grid that is farthest from the origin and valid (0.3m≤distance≤10m) is selected as the calculation point. Based on the set of calculation points, region growing clustering is performed (point distance <0.5m / angle continuity threshold ≤10°). A random sampling consistency algorithm is used for straight line fitting within a single cluster (inside point distance threshold 0.1m / inside point rate 85%). The initial line segment set with length ≥0.3m and endpoint offset <0.1m is output.

[0119] By processing the depth camera data, the accuracy of the wall recognition results is further verified, and the problems of environmental boundary fitting fracture and positioning drift caused by occlusion interference and sparsity of the depth camera point cloud are solved, thereby improving the accuracy of the wall recognition method.

[0120] Based on any of the above embodiments, in the sixth embodiment of the present application, refer to Figure 4 , Figure 4 This is a flow chart of the sixth embodiment of the wall recognition method of the present application. After step D13, steps E11 to E13 are also included:

[0121] Step E11: Based on a second length threshold, compare the initial line segment with the second length threshold.

[0122] In this embodiment, the second length threshold refers to another minimum length standard for determining the validity of the wall.

[0123] As an optional implementation, the coordinates of the start and end points of all initial line segments are traversed, the Euclidean distance of the initial line segments is calculated as the length value, and the length value is compared with the second length threshold one by one.

[0124] As an optional implementation of the comparison results, if the line segment length is greater than a threshold, it is marked as a valid wall line segment and stored in a dynamic array. If the line segment length is less than a second length threshold, it is marked as an invalid fragment, triggering the update of the fragment statistics counter, and finally outputting a valid line segment set and generating a fragment ratio report.

[0125] Step E12: If the initial line segment is greater than the second length threshold, mark the initial line segment as a wall on the map.

[0126] In this embodiment, the map refers to a dynamic raster / vector map used for robot navigation.

[0127] As an optional implementation method for initial line segment judgment, if the line segment length is greater than the threshold, it is marked as a valid wall line segment and stored in a dynamic array. If the line segment length is less than the threshold, it is marked as an invalid fragment, triggering the update of the fragment statistics counter, and finally outputting a valid initial line segment and marking the initial line segment on the map.

[0128] Step E13: Based on the initial line segment marked as a wall, the obstacle point cloud within a preset first distance threshold from one side of the robot is marked as an obstacle on the map.

[0129] In this embodiment, the initial line segment marked as a wall refers to a valid wall geometric feature written to the map after length threshold verification. The robot's preset first distance threshold refers to the dynamic monitoring range centered on the robot. The obstacle point cloud refers to the three-dimensional point set of dynamic objects identified by the depth camera. Obstacle marking refers to the conversion of the obstacle's spatial position into a map grid occupancy state.

[0130] As an optional implementation, a monitoring area is established around the robot body, obstacle point cloud data is scanned in real time, the coordinates of the obstacles are calculated through the obstacle point cloud data, and the obstacles are marked on the map.

[0131] As an optional implementation method for obstacle marking, a wall-side monitoring area is established around the robot body, and obstacle point cloud data is scanned in real time. Point sets that exist for more than 0.5 seconds and have a local density of 20 points per cubic meter are determined to be valid obstacles; the coordinates of the obstacle's centroid are calculated and mapped to the map grid, and red obstacle blocks are marked with an accuracy of 5 centimeters; the obstacle displacement speed is analyzed, and if it exceeds 0.2 meters per second, it is marked as a dynamic obstacle. The map obstacle layer is refreshed every 0.1 second and marks that have not been updated for more than 5 seconds are cleared.

[0132] As an optional implementation method of three-dimensional obstacle verification, a three-dimensional grid map is constructed in the projection area of ​​the target line segment, and dual-source fusion verification is performed on each grid: the lidar point cloud detects the vertical continuous distribution characteristics within the grid. If there are point clouds of four or more consecutive height layers, they are marked as vertical structures. The depth camera point cloud analyzes the RGB color histogram of the grid area and calculates the Bhattacharyya coefficient with the adjacent wall grid. If the Bhattacharyya coefficient is lower than the threshold, the height review is triggered; the judgment of low objects requires that the vertical discreteness of the lidar is greater than the preset discreteness and the depth camera texture discrete coefficient is less than the preset discrete coefficient. Three frames of continuous verification are performed on suspected obstacles, and the output is confirmed only after three consecutive judgments of fixed shapes.

[0133] As an optional implementation method for dynamic obstacle trajectory filtering, this method receives as input a sequence of historical obstacle centroid positions collected at fixed time intervals and outputs predicted position coordinates for a specific amount of time in the future. The linear distance difference between the actual obstacle centroid and the predicted position in the current frame is detected in real time, and the historical fluctuation amplitude of the motion direction angle is calculated. If the distance difference exceeds a set threshold and the directional fluctuation amplitude is higher than a critical value, the obstacle is marked as a strongly dynamic target and assigned a lower dynamic influence weight coefficient. Static obstacles are assigned a higher static influence weight coefficient. During the target segment fitting process, a weighted least squares optimization algorithm is used: high weight coefficients are set for static points and low weight coefficients for dynamic points, so that the sum of the weighted squares of the distances from all observation points to the target segment reaches a minimum, where the weighted contribution of static points is significantly higher than that of dynamic points. By comparing the weighted contributions, static obstacles are identified and the interference of dynamic obstacle trajectories on wall recognition is filtered out.

[0134] For example, the length of each initial line segment generated by the depth camera is calculated in real time, and valid line segments with a length ≥ the second length threshold (0.5m) are screened as walls; the endpoint coordinates of the wall segment are mapped to the navigation map through coordinate system conversion, and blue continuous marks are drawn with an accuracy of 0.02m (the endpoints of the new and old segments are fused with a confidence weight of 7:3); at the same time, with the center of the robot body as the origin, a 1.2m wide and 3m long rectangular monitoring area is delineated on the wall side, and the point set with a persistence of >0.5 seconds and a local density >20 points / 0.1m3 in the obstacle point cloud is scanned. After calculating the center of mass coordinates, it is marked as a red obstacle block with a grid accuracy of 0.05m.

[0135] By identifying obstacles by marking them and combining them with a length threshold, the wall with obstacles can be identified, solving the map navigation conflict problem caused by the marking of wall breaks and the missed detection of near-field dynamic obstacles, and improving the accuracy of the wall recognition method.

[0136] Based on any of the above embodiments, in the seventh embodiment of the present application, step S30 includes steps F11 to F13:

[0137] In step F11 , the point cloud data whose distance from the initial line segment is less than a second distance threshold is used as the wall point cloud, and the point cloud data whose distance from the initial line segment is greater than the second distance threshold is used as the obstacle point cloud.

[0138] In this embodiment, the second distance threshold refers to the maximum vertical distance for determining that the point cloud belongs to a wall.

[0139] As an optional implementation, a vertical distance calculation is performed on each standard point cloud data. The vertical distance from the standard point cloud data to all initial line segments is obtained and the minimum value is recorded. If the value is not greater than the second distance threshold and is vertically located between the two endpoints of the initial line segment, the point cloud data is classified as a wall point cloud. If the value is greater than the second distance threshold, the point cloud is classified as an obstacle point cloud.

[0140] Step F12: Project the wall point cloud onto the target line segment, cluster the projected wall point cloud, and generate a target projected line segment.

[0141] In this embodiment, projection refers to mapping a three-dimensional point along the normal of a target line segment to the linear coordinate system where the line segment is located.

[0142] As an optional implementation, by detecting the geometric matching degree of the target line segment and the target projection line segment, the endpoints of the target line segment and the target projection line segment are first determined, and the angle between the target line segment and the target projection line segment is obtained by endpoint calculation. It is verified that the angle is less than the preset angle threshold and the distance between the nearest endpoints is less than the preset distance. If the conditions are met, the target line segment and the origin of the target projection line segment coordinate system are forced to be aligned, the associated point clouds of the two are merged, and weighted fitting is performed to generate a new straight line, and the target projection line segment is output.

[0143] As an optional implementation, the wall point cloud is mapped vertically along the normal of the target line segment to the straight line where the line segment is located to obtain the projected straight line, and the angular deviation between the projected straight line and the target straight line is checked. By determining the starting points of the projected straight line and the target straight line, the angular deviation between the two is calculated. If the angular deviation is less than the preset deviation threshold, the projected straight line is used as the target projection segment.

[0144] As an optional implementation method for determining the initial line segment, if the angle deviation is greater than the preset deviation threshold, the projection angle is readjusted and re-projected. If the re-projection still does not meet the preset deviation threshold, wall recognition is restarted and wall detection failure is output.

[0145] Step F13: Project the obstacle point cloud onto the target projection line segment to generate at least one target wall line segment.

[0146] In this embodiment, merging refers to aligning the endpoints of two line segments based on geometric continuity and fusing them into a single wall model. The target wall line segment refers to the final wall representation after merging.

[0147] As an optional implementation, the obstacle point cloud is projected onto the target projection line segment, and the corresponding associated positions of the target projection line segment are disconnected by the obstacle point cloud to generate at least one target wall line segment.

[0148] As an optional implementation method of obstacle point cloud merging, based on the obstacle point cloud included on the target projection line segment, the target line segment is merged with the target projection line segment, the obstacle point cloud is merged onto the target line segment, and the target line segment is interrupted by the obstacle point cloud to display the obstacle on the target line segment.

[0149] Exemplarily, the lengths of all initial line segments are calculated in real time and compared with the second length threshold (0.5 meters). The line segments that meet the standard are selected as valid walls, the endpoint coordinates of the wall segments and their normal vectors are imported into the occupancy grid map, and continuous wall marks are drawn with a precision of five centimeters. At the same time, with the center of the robot body as the reference, rectangular monitoring areas with a preset first distance threshold (1.2 meters) are delineated on both sides of the forward direction. The point set that falls into the monitoring area in the obstacle point cloud is detected in real time (a single point appears continuously for more than 0.5 seconds). The valid obstacles are determined by the voxelized spatial occupancy rate (single grid point density > 30 points / square meter) and are marked as dynamic obstacles based on the dynamic speed vector of the obstacle (moving speed > 0.2 meters / second). Finally, the wall (blue) and obstacle (red) marking layers of the map are updated at a frequency of 20 Hz.

[0150] By using length thresholds and obstacle markings, the path planning conflicts and dynamic obstacle omissions caused by the coupled labeling of walls and obstacles in traditional methods are solved, thereby improving the accuracy of the wall recognition method.

[0151] Based on any of the above embodiments, in the eighth embodiment of the present application, after step S40, steps G11 to G13 are further included:

[0152] Step G11: Compare the edge distance between the target wall segment and one side of the robot with a preset target edge distance to generate a prediction result.

[0153] In this embodiment, the robot's side contact distance refers to the vertical distance between the outermost edge of the robot's structure and the target wall segment. The preset target contact distance refers to the ideal safe distance between the robot and the wall, as set by the navigation system. The predicted result refers to the predicted distance between the wall and the robot.

[0154] As an optional implementation, the vertical distance from the target wall segment to the robot's side contour is determined as the edge distance. This edge distance is continuously compared with a preset target edge distance, and a prediction result is generated through comparison. If the edge distance is greater than the preset target edge distance, the robot is determined to have strayed too far from the wall, and an inward compensation action instruction is generated. If the edge distance is less than the preset target edge distance, the robot is determined to have been too close to the wall, and an outward avoidance compensation instruction is generated. If the edge distance is equal to the preset target edge distance, the robot is determined to be driving normally close to the wall.

[0155] Step G12: If the prediction result shows that the edge distance is less than the preset target edge distance, an adjustment strategy corresponding to the prediction result is generated.

[0156] In this embodiment, the adjustment strategy refers to a correction action plan generated for the predicted deviation of the edge distance, which is used to guide the robot movement to eliminate the deviation.

[0157] As an optional practical method, based on the comparison result, if the edge distance is greater than the preset target edge distance, it is determined that the robot has deviated too far from the wall, and an inward compensation action instruction is generated; if the edge distance is less than the preset target edge distance, it is determined that the robot is too close to the wall, and an outward avoidance compensation instruction is generated; if the edge distance is equal to the preset target edge distance, it is determined that the robot is driving normally close to the wall.

[0158] Step G13: Adjust the path of the robot according to the adjustment strategy to keep the edge distance equal to the preset target edge distance.

[0159] In this embodiment, path adjustment refers to a motion control process that drives the robot's posture change based on an adjustment strategy.

[0160] As an optional implementation method, based on the generated adjustment strategy, the required posture change of the robot is calculated in real time. When the strategy requires inward avoidance, the path planning module immediately generates a sequence of horizontal displacement path points pointing away from the wall, and synchronously superimposes angle correction instructions to adjust the robot's heading angle so that it travels parallel to the target wall segment to keep the edge distance equal to the preset target edge distance.

[0161] As an optional implementation method for judging the validity of the target wall segment, the length value of the target wall segment is calculated through the coordinates of the two endpoints of the target wall segment, and the length value is compared with the segment threshold to determine the comparison result. If the length value is greater than the threshold, the "wall detection success" flag is output, and the target wall segment in the robot coordinate system is output, and the edge navigation module is activated. If the length is insufficient, the "wall detection failure" flag is output, and the rescanning process is triggered.

[0162] For example, the vertical distance from the right physical contour of the robot to the target wall segment is measured in real time as the actual measured value of the edge distance, and it is numerically compared with the preset target edge distance (30 cm) in a period of 20 milliseconds: if the actual measured value is less than the target value, an inward avoidance adjustment strategy is generated based on the distance deviation, and the posture offset angle (threshold 1 degree) caused by the non-orthogonality of the robot's travel direction and the wall is calculated simultaneously, and the horizontal displacement compensation and angle correction are comprehensively output. When adjusting the path, the motion controller dynamically plans the smooth path points that are offset inward according to the displacement compensation, and integrates the angle correction to drive the robot along an arc trajectory away from the wall through the left and right wheel differential control, and verifies in real time whether the adjusted edge distance returns to the target value. If the deviation still exceeds ±1 cm or the angle error is greater than 1 degree, the compensation is immediately replanned until the edge distance is stabilized and equal to the preset target value. The whole process ensures dynamic continuous tracking without control hysteresis.

[0163] By setting the preset target edge distance, the motion instability problem caused by false triggering of the edge on short walls and coordinate system drift is overcome, and the stability of the wall recognition method is improved.

[0164] Based on any of the above embodiments, in the ninth embodiment of the present application, refer to Figure 5 , Figure 5 This is a flowchart of the wall recognition method of this application. An optional implementation method of the wall recognition method of this application is to first obtain the point cloud of the depth camera's field of view that can hit the ground and the part that exceeds the robot height at the same time, remove the point cloud of other parts, and then filter out the point cloud of the ground and the point cloud that exceeds the robot height for the next step of processing. Figure 6 As shown, Figure 6 Keep a map of the region for the depth camera point cloud.

[0165] Then, the point cloud is projected onto the XY plane, and voxel filtering is performed. The filtered points are divided into angle grids with the center point of the depth camera as the coordinate origin, and then the point farthest from the origin in each angle grid is taken as the calculation point. Figure 7 As shown, Figure 7 Schematic diagram of the establishment of the angle grid and the selection of calculation points.

[0166] Next, the points are fitted with a straight line using a random sampling consistency algorithm. If a sufficiently long straight line can be formed, these points will be marked as walls and placed on the map. Figure 8 As shown, Figure 8 Schematic diagram of lidar data processing.

[0167] Then check whether there are other point clouds within a certain distance of the obtained straight line close to the robot side. If there are other point clouds, these points are considered obstacles and marked on the map. Figure 9 As shown, Figure 9Schematic diagram of map point selection and projection.

[0168] Then obtain real-time 2D lidar data, extract the point cloud data of the position where the robot intends to touch the edge, perform straight line fitting on the point cloud data using the random sampling consistency algorithm on the XY plane, and extract the line segment that finally passes through the right side of the robot. Figure 10 As shown, Figure 10 Schematic diagram of the target wall segment.

[0169] If the 2D LiDAR identifies a sufficiently long line segment, the wall and obstacle points acquired by the depth camera are converted to the LiDAR coordinate system and projected onto the line segment. The unprojected portions of the line segment are removed, and any generated obstacle points are projected onto the line segment, breaking it. The resulting processed line segment is the wall.

[0170] The present application provides a wall recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the wall recognition method in the above-mentioned embodiment one.

[0171] Reference below Figure 11 , which shows a schematic diagram of the structure of a wall recognition device suitable for implementing the embodiments of the present application. The wall recognition device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), cleaning robots, etc., as well as fixed terminals such as sweeping robots, desktop computers, etc. Figure 11 The wall recognition device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0172] like Figure 11As shown, the wall recognition device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the wall recognition device. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the wall recognition device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a wall recognition device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0173] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0174] The wall recognition device provided in this application utilizes the wall recognition method of the above-mentioned embodiment, which can solve the technical problem in the related art that, due to the limitation of the robot body size, there are blind spots in the field of view when working at close range, making it difficult to effectively recognize the wall. Compared with the existing technology, the beneficial effects of the wall recognition device provided in this application are the same as those of the wall recognition method provided in the above-mentioned embodiment, and the other technical features of the wall recognition device are the same as those disclosed in the method of the above-mentioned embodiment, and are not further described here.

[0175] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0176] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0177] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the wall recognition method in the above-mentioned embodiment.

[0178] The computer-readable storage medium provided in the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM, CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF, Radio Frequency), etc., or any suitable combination thereof.

[0179] The computer-readable storage medium may be included in the wall recognition device; or may exist independently without being assembled into the wall recognition device.

[0180] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the wall recognition device, the wall recognition device: fits the laser radar point cloud collected by the laser point cloud radar to obtain multiple fitting line segments; selects the target line segment with the longest length greater than a first length threshold among the multiple fitting line segments; projects the target point cloud associated with the position of the target line segment in the wall point cloud and obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment; if the length of the target wall line segment is greater than the line segment threshold, outputs the target wall line segment in the robot coordinate system for use by the edge fitting program.

[0181] The computer program code for performing the operations of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0182] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0183] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0184] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned wall recognition method. This computer-readable storage medium can address the technical problem in related art where, due to limitations in robot size, there are blind spots in the field of view during close-range operations, making it difficult to effectively recognize walls. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the wall recognition method provided in the aforementioned embodiment, and are not further elaborated here.

[0185] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for identifying a wall, characterized in that: Applied to a robot, the robot is provided with a laser point cloud radar and a depth camera, the angle between the midline of the field of view of the laser point cloud radar and the depth camera is a preset angle, the method includes: Fitting the laser radar point cloud collected by the laser point cloud radar to obtain multiple fitting line segments; Filtering a target line segment with a length greater than a first length threshold and the longest length among the plurality of fitted line segments; Projecting the target point cloud associated with the position of the target line segment in the wall point cloud and the obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment; If the length of the target wall line segment is greater than the line segment threshold, the target wall line segment in the robot coordinate system is output for use by the edge-fitting program.

2. The wall recognition method according to claim 1, wherein: Before the step of fitting the laser radar point cloud collected by the laser point cloud radar to obtain a plurality of fitting line segments, the method further includes: Convert the laser radar data of the current frame into the robot coordinate system to obtain coordinate point cloud data; The coordinate point cloud data is projected onto the XY plane to obtain the laser radar point cloud.

3. The wall recognition method according to claim 1, wherein: The step of fitting the laser radar point cloud collected by the laser point cloud radar to obtain a plurality of fitting line segments includes: Evenly dividing the space where the laser radar point cloud is located to determine a plurality of identical grids; Aggregating the point cloud portion within each of the grids to obtain a filtered lidar point cloud; Based on the filtered lidar point cloud, straight line fitting and cluster decomposition are performed to determine a plurality of fitting line segments.

4. The wall recognition method according to claim 1, wherein: The step of screening the target line segment having the longest length and greater than the first length threshold among the plurality of fitted line segments comprises: According to a comparison between the plurality of fitted line segments and the first length threshold, taking the plurality of fitted line segments that are greater than the first length threshold as standard line segments; The lengths of the plurality of standard line segments are compared, and the standard line segment with the longest length is selected as the target line segment.

5. The wall recognition method according to claim 1, wherein: Before the step of projecting the target point cloud associated with the position of the target line segment in the wall point cloud and the obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment, the method further includes: Filtering the depth camera data projected onto the XY plane to determine standard point cloud data corresponding to the depth camera data; Taking the center point of the depth camera as the origin, divide the angle grid and select the standard point cloud data farthest from the origin in each grid as the calculation point; A fitting clustering process is performed based on the calculated points to generate initial line segments.

6. The wall recognition method according to claim 5, characterized in that: After the step of performing fitting clustering processing according to the calculated points to generate initial line segments, the method further includes: Based on a second length threshold, comparing the initial line segment with the second length threshold; If the initial line segment is greater than the second length threshold, marking the initial line segment as a wall on the map; Based on the initial line segment marked as a wall, the obstacle point cloud within a preset first distance threshold from one side of the robot is marked on the map as an obstacle.

7. The wall recognition method according to claim 1, wherein: The step of projecting the target point cloud associated with the position of the target line segment in the wall point cloud and the obstacle point cloud collected by the depth camera onto the target line segment to determine at least one target wall line segment includes: The point cloud data whose distance from the initial line segment is less than a second distance threshold is used as the wall point cloud, and the point cloud data whose distance from the initial line segment is greater than the second distance threshold is used as the obstacle point cloud; Projecting the wall point cloud onto the target line segment, clustering the projected wall point cloud to generate a target projected line segment; The obstacle point cloud is projected onto the target projection line segment to generate at least one target wall line segment.

8. The wall recognition method according to claim 1, wherein: After the step of outputting the target wall line segment in the robot coordinate system for use by the edge-adhesion program if the length of the target wall line segment is greater than the line segment threshold, the method further includes: The prediction result is generated by comparing the edge distance between the target wall segment and one side of the robot with the preset target edge distance; If the prediction result shows that the edge distance is less than the preset target edge distance, generating an adjustment strategy corresponding to the prediction result; According to the adjustment strategy, the path of the robot is adjusted to keep the edge distance equal to the preset target edge distance.

9. A wall recognition device, characterized in that: The wall recognition device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the wall recognition method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the wall recognition method according to any one of claims 1 to 8 are implemented.

Images