Obstacle point detection method and device and storage medium

Abstract

An obstacle point detection method and device and a storage medium. The obstacle point detection method comprises: acquiring data of a historical point cloud collected in a first historical time period during traveling of a mobile device; using the data of the historical point cloud to perform plane fitting to obtain a target fitted plane under a target coordinate system; acquiring data of a current point cloud collected by the mobile device at a current moment; if the data of the current point cloud is not the data under the target coordinate system, converting the data of the current point cloud into the data under the target coordinate system; calculating a first distance between each point in the current point cloud under the target coordinate system and the target fitted plane to obtain a plurality of first distances; and determining, as obstacle points at the current moment, points corresponding to the first distances greater than a first distance threshold among the plurality of first distances. By means of the method, the present application can implement effective recognition of obstacles in a traveling path of a mobile device.

Description

Obstacle point detection method, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on February 2, 2024, with application number 2024101500501, and invention name “Obstacle point detection method, device and storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of data processing technology, and more particularly to a method and device for detecting an obstacle point, and a computer-readable storage medium. Background Art

[0003] In industries such as robotics and autonomous driving, it is necessary to detect in real time whether there are obstacles in the path of mobile devices. Once an obstacle is detected in its path, it is necessary to respond in a timely manner, such as by evasive maneuvers, to avoid the obstacle. If the obstacle cannot be detected in a timely and accurate manner for avoidance, accidents will occur.

[0004] Some low obstacles are smaller than the height of the mobile device. If they fall within the blind spot of the obstacle detection sensor (i.e., the primary sensor) in the mobile device, they may not be detected in a timely and effective manner. Therefore, there is currently a lack of an effective method for detecting low obstacles. Summary of the Invention

[0005] In view of the above problems, embodiments of the present application provide an obstacle point detection method, apparatus, device and storage medium to solve the problem in the prior art that low obstacles in the path of a mobile device cannot be effectively detected.

[0006] According to one aspect of an embodiment of the present application, a method for detecting obstacle points is provided, the method being used to detect obstacle points in a travel path of a mobile device, the method comprising: obtaining data of a historical point cloud collected within a first historical period during the travel of the mobile device; performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system; obtaining data of a current point cloud collected by the mobile device at a current moment; if the data of the current point cloud is not data in the target coordinate system, converting the data of the current point cloud into data in the target coordinate system; calculating a first distance between each point in the current point cloud in the target coordinate system and the target fitting plane to obtain multiple first distances, wherein each point corresponds to a first distance; and determining a point corresponding to a first distance greater than a first distance threshold among the multiple first distances as an obstacle point at the current moment.

[0007] In an optional manner, performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system includes: performing plane fitting using data of multiple points in the historical point cloud to obtain a fitting plane in the target coordinate system; calculating a second distance between each of the multiple points used to fit the fitting plane and the fitting plane to obtain multiple second distances, wherein each point in the multiple points corresponds to a second distance; determining a point corresponding to a second distance less than a second distance threshold among the multiple second distances as an inlier; calculating a ratio of the number of the inliers to the number of the multiple points used to fit the fitting plane to obtain an inlier rate; if the inlier rate is less than an inlier rate threshold, then: repeating the above steps until the inlier rate is greater than or equal to the inlier rate threshold, wherein the data of the multiple points used in each plane fitting are not exactly the same when the above steps are repeated; or the number of times the above steps are repeated reaches a number threshold, and determining the fitting plane corresponding to the maximum inlier rate as the target fitting plane; if the inlier rate is greater than or equal to the inlier rate threshold, then determining the fitting plane as the target fitting plane.

[0008] In an optional manner, when the step of performing plane fitting using the data of multiple points in the historical point cloud to obtain the fitting plane in the target coordinate system is performed for the Nth time, the data of the multiple points used for plane fitting are the data of multiple inner points among the multiple points used for the N-1th plane fitting, where N is a positive integer and N≥2.

[0009] In an optional manner, the data of the historical point cloud and the data of the current point cloud are data obtained through a depth sensor in the mobile device, and the target coordinate system is a coordinate system constructed with a target point in the mobile device as the origin, wherein the target point is a point that does not belong to the depth sensor.

[0010] In an optional manner, the data of the historical point cloud is P k ={P k0 ,P k1 ,...,P kn}, where P k is the data in the depth sensor coordinate system, and the posture of the mobile device corresponding to each point in the historical point cloud is T k ={T k0 ,T k1 ,...,T kn}, wherein the use of the historical point cloud data to perform plane fitting to obtain the target fitting plane in the target coordinate system includes: using the formula P k Converted into historical point cloud data in the target coordinate system, where is the historical point cloud data in the target coordinate system, T curr is the posture of the mobile device at the current moment, The external parameters for converting the depth sensor coordinate system to the target coordinate system; using the data of the historical point cloud in the target coordinate system Perform plane fitting to obtain the target fitting plane in the target coordinate system; if the data of the current point cloud is not the data in the target coordinate system, convert the data of the current point cloud into the data in the target coordinate system, including: if the data of the current point cloud is the data in the depth sensor coordinate system, use the formula The data of the current point cloud in the depth sensor coordinate system is converted into data in the target coordinate system, wherein: is the data of the current point cloud in the target coordinate system, P curr The data of the current point cloud in the depth sensor coordinate system is obtained by the depth sensor.

[0011] In an optional embodiment, the method further includes: obtaining an obstacle point cloud within a second historical period during the movement of the mobile device, wherein the obstacle point cloud is a collection of obstacle points at each moment within the second historical period; and determining each obstacle point in the obstacle point cloud within the second historical period as an obstacle point at the current moment.

[0012] In an optional manner, the data of the obstacle point cloud in the second historical period is P k '={P k0 ',P k1 ',...,P kn '}, the P k ' is the data in the target coordinate system, and the posture of the mobile device corresponding to each point in the obstacle point cloud in the second historical period is T k '={T k0 ',T k1 ',...,T kn '}, wherein the determining each obstacle point in the obstacle point cloud within the second historical period as the obstacle point at the current moment includes: using the formula The obstacle point cloud data in the second historical period is converted into the point cloud data at the current moment, wherein: is the data of the obstacle point cloud at the current moment in the converted target coordinate system, T curr is the posture of the mobile device at the current moment; The obstacle point in is determined as the obstacle point at the current moment.

[0013] In an optional manner, the mobile device includes a depth sensor, and the historical point cloud data and the current point cloud data are both coordinate data of detection points within a detection blind spot of a main sensor of the mobile device obtained by the depth sensor.

[0014] According to another aspect of an embodiment of the present application, a device for detecting obstacle points is provided, the device being used to detect obstacle points in a travel path of a mobile device, the device comprising: a first acquisition module for acquiring data of a historical point cloud collected within a first historical period during the travel of the mobile device; a plane fitting module for performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system; a second acquisition module for acquiring data of a current point cloud collected by the mobile device at a current moment; a conversion module for converting the data of the current point cloud into data in the target coordinate system if the data of the current point cloud is not in the target coordinate system; a calculation module for calculating a first distance between each point in the current point cloud in the target coordinate system and the target fitting plane to obtain multiple first distances, wherein each point corresponds to a first distance; and a determination module for determining a point corresponding to a first distance greater than a first distance threshold among the multiple first distances as an obstacle point at the current moment.

[0015] According to another aspect of an embodiment of the present application, an obstacle point detection device is provided, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to perform the operations of the obstacle point detection method described above.

[0016] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the storage medium stores executable instructions, and the executable instructions, when run, execute the operations of the above-mentioned obstacle point detection method.

[0017] In the embodiment of the present application, both the historical point cloud data and the current point cloud data are obtained by using the depth sensor to obtain the coordinate data of the detection points within the detection blind spot of the main sensor. Since the target fitting plane is obtained by plane fitting using the data of multiple points in the mobile device's travel path collected in a historical period close to the current moment, the obtained target fitting plane is similar to the ground in the mobile device's current travel path. Then, by using the distance between the points collected at the current moment and the target fitting plane, the points outside the plane can be accurately identified. The points outside the plane are obstacle points. For example, the points of obstacles on the ground and the points of ditches under the ground are both at a certain distance from the ground. These obstacle points are all outside the ground.

[0018] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0020] FIG1 is a schematic diagram showing a flow chart of an obstacle point detection method provided in an embodiment of the present application;

[0021] FIG2 is a schematic diagram showing a laser radar in a mobile device according to an embodiment of the present application emitting laser light toward the ground;

[0022] FIG3 shows a flow chart of sub-steps of step 120 in FIG1 ;

[0023] FIG4 shows a schematic structural diagram of an obstacle point detection device provided in an embodiment of the present application;

[0024] FIG5 shows a schematic structural diagram of an obstacle point detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0026] In the embodiments of this application, mobile devices refer to devices that autonomously plan and move their paths, such as robots (e.g., robot vacuums) and unmanned vehicles. Because mobile devices autonomously plan their paths, they need to detect obstacles in their paths in real time. If obstacles are present, they need to replan their paths to avoid them and prevent accidents.

[0027] The inventors of this application discovered that in order to detect low obstacles located within the blind spot of the primary sensor in a mobile device, a single-line laser radar can be installed at an angle (for example, 20 cm above the ground, tilted downward by 15° to 20°) in the mobile device, and low obstacles can be detected using distance-based detection methods or template matching-based detection methods. However, when the pitch angle of the mobile device fluctuates, these two methods cannot effectively distinguish between obstacles and the ground in the path of travel, resulting in missed or false detection of obstacles.

[0028] Therefore, based on the above considerations, in order to effectively detect low obstacles, this application proposes a method for detecting obstacle points, where an obstacle point is a point that is an obstacle. This method first obtains historical point cloud data collected over a period of time during the movement of a mobile device. The historical point cloud data is the coordinate data of multiple detection points in the mobile device's path obtained by a depth sensor. A plane fitting is then performed using the historical point cloud data to obtain a fitted plane. Points in the point cloud collected at the current moment that lie outside the fitted plane are then identified as obstacle points. Since mobile devices primarily move on the ground, by performing plane fitting using historical point cloud data from a period of time, the obtained fitted plane can be used to represent the ground on which the mobile device is traveling. This allows the fitted plane to effectively detect whether an obstacle point exists in the point cloud collected at the current moment.

[0029] FIG1 shows a flow chart of an obstacle point detection method provided in an embodiment of the present application, the method is used to detect obstacle points in the path of a mobile device, and the method is executed by a terminal device, wherein the terminal device may be a mobile device, a controller in a mobile device, or other terminal device connected to the mobile device for communication. The terminal device may be a terminal device including one or more processors, the processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present application, which is not limited here. The one or more processors included in the terminal device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs, which are not limited here. As shown in FIG1 , the method includes the following steps:

[0030] Step 110: Acquire historical laser point cloud data collected during a first historical period of time during the movement of the mobile device.

[0031] It is worth noting that in order to obtain historical point cloud data with higher accuracy, a depth sensor can be installed in the mobile device to obtain the coordinate data of the detection points within the blind spot of the mobile device's main sensor through the depth sensor. The depth sensor can be a lidar or a depth camera. The following description only uses the lidar as an example. If a depth camera is used to obtain the coordinate data of the detection points within the blind spot of the mobile device's main sensor, the data collected in step 110 is camera point cloud data rather than laser point cloud data. The laser point clouds mentioned below are also camera point clouds, and the laser points are all camera points.

[0032] In this step, the data of the historical laser point cloud collected in the first historical period refers to the coordinate data of multiple laser points collected at multiple moments in the past period of time. For example, it can be the coordinates of multiple laser points collected at multiple moments within 3 to 5 seconds past the current moment. Preferably, the first historical period is a historical period close to the current moment. Among them, by emitting a laser to the detection point through the laser radar, the coordinates of the detection point can be determined based on the signal reflected from the detection point received by the laser radar. It is worth noting that in the embodiment of the present application, preferably, the data of the historical laser point cloud is data obtained by the laser radar in the mobile device.

[0033] In order to better illustrate the method of obtaining historical laser point cloud data, Figure 2 shows a schematic diagram of the laser radar in the mobile device provided by the embodiment of the present application emitting laser to the ground, wherein (a) in Figure 2 is a side view of the laser radar emitting laser to the ground, and (b) is a top view of the laser radar emitting laser to the ground. As shown in (a) in Figure 2, the laser radar 11 in the mobile device 10 emits lasers toward multiple detection points on the ground (indicated by dotted lines in the figure). After the laser reaches the detection point, the coordinate data of the detection point can be determined based on the signal reflected from the detection point, and the position information of the detection point is obtained. If there is an obstacle 20 on the ground, after the laser radar emits laser to the ground, the emitted laser cannot reach the ground where the obstacle 20 is present, but reaches the obstacle 20. At this time, the coordinate data of the point on the surface of the obstacle 20 is obtained.

[0034] It is understandable that since the laser radar emits lasers to multiple detection points on the ground within a certain range to obtain historical laser point cloud data, if there are low obstacles on the ground, most of the laser points included in the historical laser point cloud obtained belong to laser points on the ground, and the proportion of obstacle points is relatively small.

[0035] Step 120: Perform plane fitting using the historical laser point cloud data to obtain a target fitting plane in the target coordinate system.

[0036] The mobile device coordinate system refers to a coordinate system created with a point in the mobile device as its origin and rotates around that origin. The origin is a point that is not part of the LiDAR. The LiDAR coordinate system refers to a coordinate system created with a point in the LiDAR as its origin. It is similar to the mobile device coordinate system, except for its different origin. In the embodiments of the present application, the target coordinate system can be set as needed and can be either a world coordinate system or a mobile device coordinate system.

[0037] The laser point cloud data usually obtained by laser radar collection is data in the laser radar coordinate system. After the historical laser point cloud data in the laser radar coordinate system collected in step 110 is converted into data in the mobile device coordinate system or the world coordinate system, the converted data can be used for plane fitting to obtain the target fitting plane in the target coordinate system.

[0038] For example, if the target coordinate system is the world coordinate system and the historical laser point cloud data is the data in the laser radar coordinate system, the formula The historical laser point cloud data in the laser radar coordinate system is converted into the historical laser point cloud data in the world coordinate system, and then the historical laser point cloud data in the world coordinate system is used to perform plane fitting to obtain the target fitting plane in the world coordinate system. is the historical laser point cloud data in the world coordinate system, P k ={P k0 ,P k1 ,...,P kn}, P k is the data of each laser point in the historical laser point cloud in the laser radar coordinate system, T k ={T k0 ,T k1 ,...,T kn}, T k is the posture of the mobile device corresponding to each laser point in the historical laser point cloud, The external parameters for converting the LiDAR coordinate system to the mobile device coordinate system include the rotation matrix and translation vector.

[0039] The plane is expressed as A*x+B*y+C*z+D=0, where A, B, C, and D are parameters to be determined, and x, y, and z are variables. The target fitting plane can be obtained by substituting the coordinate data of at least four laser points in the historical laser point cloud collected in step 110 into the above equation.

[0040] It is understandable that if the historical laser point cloud includes obstacle points, then if the obstacle points account for a large proportion of the laser point data used for plane fitting, the accuracy of the obtained target fitting plane will be reduced. Since most laser points in the historical laser point cloud are points on the ground, within a certain range, the more laser point data is used for plane fitting, the smaller the proportion of obstacle points included in the laser points used for plane fitting, and the higher the accuracy of the obtained target fitting plane. In order to improve the accuracy of the obtained fitting plane, the plane fitting can be performed using the data of all historical laser point clouds in step 110.

[0041] Step 130: Acquire the current laser point cloud data collected by the mobile device at the current moment.

[0042] Unless otherwise specified, the current time in the entire text refers to the same time.

[0043] Step 140: Determine whether the current laser point cloud data is data in the target coordinate system. If yes, go to step 160; if not, go to step 150.

[0044] If the target fitting plane and the data of the current laser point cloud are not in the same coordinate system, the target fitting plane cannot be used to determine whether there are laser points in the current laser point cloud that are outside the target fitting plane.

[0045] Step 150: Convert the current laser point cloud data into data in the target coordinate system.

[0046] Among them, if the target coordinate system is the mobile device coordinate system, and the current laser point cloud data is in the laser radar coordinate system, then the current laser point cloud data needs to be converted to the mobile device coordinate system. If the target coordinate system and / or laser point cloud data are in other coordinate systems, the same is true, and examples are not given here. For example, if the target coordinate system is the world coordinate system, and the current laser point cloud data is in the laser radar coordinate system, then the formula Convert the current laser point cloud data in the laser radar coordinate system into the data in the world coordinate system, where P w is the data of the current laser point cloud in the world coordinate system, is the position of the mobile device in the world coordinate system, including the rotation matrix and translation vector, P curr The data of the current laser point cloud in the laser radar coordinate system.

[0047] Step 160: Calculate the first distance between each laser point in the current laser point cloud and the target fitting plane in the target coordinate system to obtain a plurality of first distances, wherein each laser point corresponds to a first distance.

[0048] If the current laser point cloud includes m laser points, the first distances between the m laser points and the fitting plane are calculated respectively to obtain m first distances, and each of the m laser points corresponds to one of the m first distances, where m is a positive integer and m≥2.

[0049] Step 170: Determine the laser point corresponding to the first distance greater than the first distance threshold among the multiple first distances as the obstacle point at the current moment.

[0050] The first distance threshold can be set as needed. For example, the first distance threshold can range from 5 cm to 10 cm, and can be set to 5 cm, 8 cm, or 10 cm, etc. Specifically, if the mobile device is a robot and the robot chassis height is 10 cm, then obstacles that are much smaller than the robot chassis height will not affect the robot's normal movement, meaning the robot does not need to avoid these obstacles. Therefore, the first distance threshold can range from 5 cm to 8 cm. If the first distances corresponding to some of the m laser points are greater than the first distance threshold, it indicates that the objects corresponding to these laser points are at a certain distance from the ground and are located outside the ground, thus representing obstacle points. For example, the points of obstacles located on the ground and the points of ditch located underground are both at a certain distance from the ground and are located outside the ground.

[0051] In the embodiment of the present application, since the target fitting plane is obtained by plane fitting using data from multiple laser points in the mobile device's travel path collected within a historical period close to the current moment, the obtained target fitting plane is approximately similar to the ground in the mobile device's current travel path. Furthermore, through the distance between the laser points collected at the current moment and the target fitting plane, points outside the plane can be accurately identified, and points outside the plane are obstacle points.

[0052] In some embodiments, to avoid data redundancy, which would result in a large amount of computation and reduce the efficiency of obstacle detection, preferably, in the direction of movement of the mobile device, only the laser point cloud data in front of the mobile device is collected. That is, the laser radar emits laser light to the ground within a certain angle range in front of it, thereby obtaining the historical laser point cloud data and the current laser point cloud data within this range. For example, the maximum range of the laser radar to emit laser light in the front is 1 to 3 times the width of the mobile device. In the direction of movement of the mobile device, since the mobile device moves forward according to its planned path, obstacles behind it will not affect its forward movement. Therefore, it does not need to detect obstacles behind it for obstacle avoidance.

[0053] FIG3 shows a flow chart of sub-steps of step 120 in FIG1 . As shown in FIG3 , step 120 includes:

[0054] Step 121: performing plane fitting using the data of multiple laser points in the historical laser point cloud to obtain a fitting plane in the target coordinate system.

[0055] Among them, this step is similar to step 120. The principle and implementation method of this step can be referred to step 120 and will not be repeated here.

[0056] Step 122: Calculating a second distance between each of the plurality of laser points used to fit the fitting plane and the fitting plane to obtain a plurality of second distances, wherein each of the plurality of laser points corresponds to a second distance.

[0057] Among them, if the historical laser point cloud includes M laser points, and the data of M′ laser points in the historical laser point cloud are used to perform plane fitting in step 121, then in this step, the second distances between these M′ laser points and the fitting plane obtained in step 121 are calculated respectively to obtain M′ second distances, and each laser point in these M′ laser points corresponds to one of the M′ second distances, M′ is a positive integer, and M′≤M.

[0058] Step 123: Determine the laser point corresponding to the second distance smaller than the second distance threshold among the plurality of second distances as an interior point.

[0059] The second distance threshold may be set as needed. For example, the second distance threshold may be in the range of 3 to 6 cm, and may be set to 3 cm, 5 cm, or 6 cm.

[0060] Step 124: Calculate the ratio of the number of inliers to the number of laser points used to fit the fitting plane to obtain an inlier rate.

[0061] If there are k′ inliers among the M′ laser points used for plane fitting, the inlier rate in this step is k′ / M′, where k′ is a positive integer and k′≤M′.

[0062] Step 125: Determine whether the inlier rate is less than the inlier rate threshold. If yes, go to step 126; if not, go to step 128.

[0063] The inlier rate may be set as needed. For example, the inlier rate threshold may range from 80% to 90%, and may be set to 80%, 85%, or 90%.

[0064] If the inlier rate is small, it means that there are multiple laser points outside the obtained fitting plane among the M′ laser points used for plane fitting. That is, these laser points may be obstacles. Then the accuracy of the obtained fitting plane is low, and the obtained fitting plane cannot be used to accurately represent the ground in the mobile device's path.

[0065] Step 126: Determine whether the number of times the above steps are repeated reaches the threshold number. If yes, go to step 127; if not, go to step 121.

[0066] If the inlier rate is low, iterative plane fitting is required. That is, steps 121 to 125 are repeated to gradually reduce the inlier rate of the obtained fitted plane until the inlier rate is greater than or equal to the inlier rate threshold or the number of repeated plane fitting reaches the number threshold, thereby obtaining a fitted plane with higher accuracy. The number threshold can be set as needed, for example, to 3, 5, or 7 times.

[0067] It is worth noting that if the inlier rate is less than the inlier rate threshold, and the number of times step 121 is repeated does not reach the number threshold, then the data of the multiple laser points used in the repeated plane fitting are not completely the same as the data of the multiple laser points used in any previous plane fitting. For example, if the historical laser point cloud includes 100 laser points, the corresponding serial numbers are 1-100 respectively. If the plane fitting is performed using the data of the laser points with serial numbers 1-80 when step 121 is performed for the first time in the embodiment of the present application, then the serial numbers of the multiple laser points used when step 121 is performed for the second time are not completely 1-80. For example, they can be laser points with serial numbers 5-90, or laser points with serial numbers 10-100, etc. When step 121 is performed for the jth time, the laser points used for plane fitting are not completely the same as the laser points used for the previous j-1 times of plane fitting, where j is a positive integer and j≥2.

[0068] Step 127: Determine the fitting plane corresponding to the maximum inlier rate as the target fitting plane.

[0069] It can be understood that each time step 121 is executed for plane fitting, the inlier rate of the fitted plane will be calculated accordingly, that is, if step 121 is executed j times, j fitting planes and j inlier rates will be obtained accordingly, and each fitting plane corresponds to an inlier rate.

[0070] Among them, if the inlier rate corresponding to each fitting plane obtained after repeating step 121 for the number of times reaches the number threshold is less than the inlier rate threshold, the fitting plane corresponding to the largest inlier rate is determined as the target fitting plane, thereby improving the accuracy of the obtained fitting plane.

[0071] Step 128: Determine the fitting plane as the target fitting plane.

[0072] If the inlier rate is greater than or equal to the inlier rate threshold, the fitting plane corresponding to the inlier rate is determined as the target fitting plane.

[0073] Since historical laser point clouds may include obstacle points, if the data of multiple laser points used for plane fitting includes data of a large number of obstacle points, the accuracy of the obtained fitted plane will be low, that is, the obtained fitted plane cannot accurately represent the ground in the mobile device's path. Therefore, if the fitted plane is subsequently used to determine the obstacle points at the current moment, the accuracy of the determined obstacle points will be low. Therefore, in an embodiment of the present application, a plane is fitted using the data of multiple laser points in a historical laser point cloud. After obtaining the fitted plane, the inlier rate of the obtained fitted plane is calculated to determine the accuracy of the obtained fitted plane based on the inlier rate. If the inlier rate does not meet the requirements, the plane fitting is iteratively performed to gradually improve the inlier rate, thereby obtaining a target fitted plane with higher accuracy.

[0074] To improve the accuracy of the obtained fitting plane, the present embodiment provides a method for determining the target fitting plane. Based on the embodiment provided in FIG3 , in the present embodiment, when step 121 is performed for the Nth time, the data of the multiple laser points used for plane fitting is the data of the multiple inner points among the multiple laser points used for the plane fitting for the N-1th time, where N is a positive integer and N ≥ 2.

[0075] In an embodiment of the present application, if the inlier rate of the fitting plane obtained by executing step 121 for the first time is less than the inlier rate threshold, and the number threshold is greater than 1, then when step 121 is executed for the second time, the data of the multiple laser points used for plane fitting are the data of all the inliers in the multiple laser points used for plane fitting for the first time, and so on, until the inlier rate of the obtained fitting plane is greater than or equal to the inlier rate threshold or the number of times step 121 is repeated reaches the number threshold, thereby obtaining a target fitting plane with higher accuracy.

[0076] For example, if the laser point cloud data used for plane fitting for the first time includes data of 100 laser points, the 100 laser points include 80 inliers, and the inlier rate is 80%; then when step 121 is executed for the second time, the plane is fitted using the data of these 80 laser points. If these 80 laser points include 75 inliers, the inlier rate is 93.75%, and so on, thereby gradually improving the inlier rate of the obtained fitting plane, and thus improving the accuracy of the obtained fitting plane.

[0077] After an obstacle point is detected, in order to enable the mobile device to make timely response actions such as avoidance, based on the embodiment provided in FIG1 , in the embodiment of the present application, the target coordinate system is the mobile device coordinate system.

[0078] Among them, after the mobile device recognizes the existence of an obstacle point in the travel path, it needs to further determine the distance between the obstacle and itself, as well as the direction of the obstacle, in order to further plan the path to avoid the obstacle. For example, if the laser radar is set on the leftmost side of the mobile device, and the mobile device is currently turning to the left, if the coordinate data of the obstacle point determined is the data in the laser radar coordinate system, the obstacle point is located directly in front of the laser radar and on the left side of the mobile device, and if the mobile device will collide with the obstacle point while turning left along the originally planned path, then the mobile device needs to re-plan the path to avoid the obstacle. However, if the mobile device directly uses the coordinate data of the obstacle point in the laser radar coordinate system to plan the path to avoid the obstacle point, it will mistakenly believe that the obstacle point is located directly in front of it, resulting in low accuracy of the planned path and failure to avoid the obstacle.

[0079] Therefore, in the embodiment of the present application, by setting the target coordinate system as the mobile device coordinate system, the coordinate data of the determined obstacle point is also data under the mobile device coordinate system. Therefore, the mobile device performs path planning based on the coordinate data of the determined obstacle point, thereby improving the accuracy of the obtained path and effectively achieving obstacle avoidance.

[0080] Furthermore, if the coordinate data of the determined obstacle point is not in the mobile device coordinate system, converting it to the mobile device coordinate system before performing path planning will result in a low efficiency of the resulting path. If the obstacle is small, it is easy to fail to avoid the obstacle in time, leading to accidents. Therefore, in the embodiment of the present application, by setting the target coordinate system to the mobile device coordinate system, the mobile device can avoid the obstacle in a timely and effective manner after detecting the obstacle point, thus preventing accidents.

[0081] In order to obtain the target fitting plane in the target coordinate system with higher accuracy, and the data of the current laser point cloud in the target coordinate system, based on the above embodiment, in the embodiment of the present application, the data of the historical laser point cloud is P k ={P k0 ,P k1 ,...,P kn}, where P k is the data in the laser radar coordinate system, and the posture of the mobile device corresponding to each laser point in the historical laser point cloud is T k ={T k0 ,T k1 ,...,T kn}, wherein step 120 includes:

[0082] Step a1: Use the formula P k Converted into historical laser point cloud data in the target coordinate system, where is the historical laser point cloud data in the target coordinate system, T curr is the posture of the mobile device at the current moment, It is the external parameter for converting the lidar coordinate system to the target coordinate system.

[0083] Among them, P k0 、P k1 ,……,P kn Respectively represent the coordinate data corresponding to each laser point in the historical laser point cloud. k0 、T k1 ,……,T kn They represent the posture of the mobile device corresponding to each laser point in the historical laser point cloud. k0 The corresponding posture of the mobile device is T k0 , indicating that the laser radar detects the laser point P k0 When the posture of the mobile device is T k0 , other laser points are similar and will not be described in detail here. It is understood that if the laser radar is a single-line laser radar, the coordinate data of the laser point is two-dimensional data; if the laser radar is a multi-line laser radar, the coordinate data of the laser point is three-dimensional data. Indicates T curr The inverse of a matrix.

[0084] Step a2: Using historical laser point cloud data in the target coordinate system Perform plane fitting to obtain the target fitting plane in the target coordinate system.

[0085] This step is similar to step 120 and will not be repeated here.

[0086] Step 150 includes: if the current laser point cloud data is data in the laser radar coordinate system, then use the formula Convert the current laser point cloud data in the laser radar coordinate system into the data in the target coordinate system, where: is the data of the current laser point cloud in the target coordinate system, P curr It is the data of the current laser point cloud in the laser radar coordinate system obtained by the laser radar.

[0087] In an embodiment of the present application, since the laser point cloud data obtained by the laser radar is data in the laser radar coordinate system, by utilizing the external parameters of the laser radar coordinate system converted to the mobile device coordinate system, as well as the posture of the mobile device, the laser point cloud data in the laser radar coordinate system can be accurately converted into laser point cloud data in the mobile device coordinate system, thereby improving the accuracy of the target fitting plane obtained in the mobile device coordinate system.

[0088] In order to avoid misjudging the obstacle point, based on the embodiment provided in FIG1 , in the embodiment of the present application, the obstacle point detection method further includes the following steps:

[0089] Step b1: Obtain an obstacle point cloud in a second historical period during the movement of the mobile device, wherein the obstacle point cloud is a collection of obstacle points at each moment in the second historical period.

[0090] The second historical period refers to a period of time in the past that is relatively close to the current moment, for example, the past 3 to 5 seconds. The obstacle point detection method provided in Figure 1 can be used to detect obstacle points within the second historical period to obtain an obstacle point cloud. The second historical period can be the same as, different from, or partially the same as the first historical period.

[0091] Step b2: Determine each obstacle point in the obstacle point cloud within the second historical period as an obstacle point at the current moment.

[0092] Among them, after the robot detects an obstacle point, it takes a certain amount of time to react before it can perform obstacle avoidance action. If only the obstacle point in the laser point cloud collected at the current moment is determined as the obstacle point at the current moment and obstacle avoidance action is performed, if the obstacle is small, then since the mobile device is constantly moving, it is very likely that the obstacle point detected at the current moment cannot be detected at the next moment, causing the mobile device to mistakenly believe that the obstacle has been cleared and no obstacle avoidance action is required, which will lead to an accident.

[0093] For example, if an autonomous vehicle detects an obstacle and immediately stops to avoid it, it may continue to slide forward for a distance due to inertia. If the obstacle is small, the vehicle may slide forward for a distance and then fall out of its detection range. The vehicle may then fail to detect the obstacle and mistakenly believe it has been cleared, continuing to move forward and potentially causing an accident.

[0094] Therefore, in an embodiment of the present application, based on the obstacle points detected at the current moment, the obstacle points detected in the past period are also used as the obstacle points at the current moment, so that the mobile device can take obstacle avoidance actions and replan the moving route, thereby achieving obstacle avoidance and avoiding the above situation.

[0095] On the basis of the above embodiment, in the embodiment of the present application, the data of the obstacle point cloud in the second historical period is P k '={P k0 ',P k1 ',...,P kn '}(including n+1 obstacle points), P k ' is the data in the target coordinate system, and the posture of the mobile device corresponding to each laser point in the obstacle point cloud in the second historical period is Tk '={T k0 ',T k1 ',...,T kn '}, wherein step b2 includes:

[0096] Step b21: Using the formula The obstacle point cloud data in the second historical period is converted into the point cloud data at the current moment, where: is the obstacle point cloud data at the current moment in the converted target coordinate system, T curr The posture of the device at the current moment.

[0097] Since the mobile device is constantly moving, the data of the obstacle points detected in the past period is determined based on the past posture of the mobile device. Therefore, the data of the obstacle points in the second historical period needs to be converted into point cloud data at the current moment. k0 '、P k1 ', ..., P kn 'respectively represent the coordinate data corresponding to each obstacle point in the obstacle point cloud in the second historical period. k0 '、T k1 ', ..., T kn 'represents the posture of the mobile device corresponding to each obstacle point. Specifically, the laser point P k0 'The corresponding posture of the mobile device is T k0 ', indicating that the laser radar detects the laser point P k0 ', the posture of the mobile device is T k0 ', other obstacle points are similar and will not be introduced in detail here.

[0098] Step b22: The obstacle point in is determined as the obstacle point at the current moment.

[0099] In this step, The n+1 obstacle points included in are determined as the obstacle points at the current moment, and these obstacle points are added to the obstacle point set determined based on the laser point cloud data at the current moment.

[0100] In an embodiment of the present application, by utilizing the coordinate data of each obstacle point in the second historical period and the corresponding posture of the mobile device and the posture of the mobile device at the current moment, the data of the obstacle points in the second historical period is converted into the coordinate data of the obstacle points at the current moment, thereby improving the accuracy of the obtained obstacle point data.

[0101] Figure 4 shows a schematic diagram of the structure of the obstacle detection device provided by an embodiment of the present application. As shown in Figure 4, the obstacle detection device 200 includes: a first acquisition module 201, a plane fitting module 202, a second acquisition module 203, a conversion module 204, a calculation module 205, and a determination module 206.

[0102] Among them, the first acquisition module 201 is used to obtain the data of the historical laser point cloud collected during the first historical period of time during the movement of the mobile device. The plane fitting module 202 is used to use the data of the historical laser point cloud to perform plane fitting to obtain the target fitting plane in the target coordinate system. The second acquisition module 203 is used to obtain the data of the current laser point cloud collected by the mobile device at the current moment. The conversion module 204 is used to convert the data of the current laser point cloud into data in the target coordinate system if the data of the current laser point cloud is not in the target coordinate system. The calculation module 205 is used to calculate the first distance between each laser point in the current laser point cloud and the target fitting plane in the target coordinate system to obtain multiple first distances, wherein each laser point corresponds to a first distance. The determination module 206 is used to determine the laser point corresponding to the first distance greater than the first distance threshold among the multiple first distances as the obstacle point at the current moment.

[0103] The obstacle point detection device provided in this embodiment is used to implement the technical solution of the obstacle point detection method in the aforementioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0104] It is worth noting that the obstacle point detection device provided in this embodiment also includes other modules for executing the various steps of the above-mentioned obstacle point detection method embodiment, which will not be described one by one here.

[0105] FIG5 shows a schematic structural diagram of an obstacle point detection device provided in an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the obstacle point detection device.

[0106] As shown in FIG. 5 , the obstacle point detection device may include: a processor 302 , a communications interface 304 , a memory 306 , and a communication bus 308 .

[0107] Processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308. Communication interface 304 is used to communicate with other devices, such as clients or other server network elements. Processor 302 is used to execute program 310, which may specifically perform the steps described in the above-mentioned embodiment of the obstacle detection method.

[0108] Specifically, the program 310 may include program code including computer-executable instructions.

[0109] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the obstacle detection device may be of the same type, such as one or more CPUs, or may be of different types, such as one or more CPUs and one or more ASICs.

[0110] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0111] An embodiment of the present application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed on an obstacle point detection device, the obstacle point detection device executes the obstacle point detection method in any of the above method embodiments.

[0112] An embodiment of the present application provides a computer program that can be invoked by a processor to enable an obstacle point detection device to perform the obstacle point detection method in any of the above method embodiments.

[0113] An embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed on a computer, the computer executes the obstacle point detection method in any of the above method embodiments.

[0114] In the several embodiments provided in this application, if any function is implemented in the form of a software function module / unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of this application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or other electronic device) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: various media that can store computer program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] The algorithm or demonstration provided here are not inherently relevant to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the present application.

[0116] It should be noted that the above embodiments illustrate rather than limit the present application, and that a person skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In claims that list several means, several units or modules of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

[0117] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting an obstacle point, characterized in that: The method is used to detect an obstacle point in a travel path of a mobile device, and the method comprises: Acquire historical point cloud data collected during a first historical period of time during the movement of the mobile device; Performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system; Acquire the current point cloud data collected by the mobile device at the current moment; If the data of the current point cloud is not data in the target coordinate system, converting the data of the current point cloud into data in the target coordinate system; Calculating a first distance between each point in the current point cloud and the target fitting plane in the target coordinate system to obtain a plurality of first distances, wherein each point corresponds to a first distance; A point corresponding to a first distance greater than a first distance threshold among the multiple first distances is determined as the obstacle point at the current moment.

2. The method according to claim 1, characterized in that The performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system includes: Performing plane fitting using data of multiple points in the historical point cloud to obtain a fitting plane in the target coordinate system; Calculating a second distance between each of a plurality of points used to fit the fitting plane and the fitting plane to obtain a plurality of second distances, wherein each of the plurality of points corresponds to a second distance; Determine a point corresponding to a second distance smaller than a second distance threshold among the plurality of second distances as an inlier; Calculating a ratio of the number of the inliers to the number of the plurality of points used to fit the fitting plane to obtain an inlier rate; If the inlier rate is less than the inlier rate threshold, then: repeating the above steps until the inlier rate is greater than or equal to the inlier rate threshold, wherein when the above steps are repeated, the data of the multiple points used in each plane fitting are not exactly the same; or the above steps are repeated the number of times reaching the number threshold, and the fitting plane corresponding to the maximum inlier rate is determined as the target fitting plane; If the inlier rate is greater than or equal to the inlier rate threshold, the fitting plane is determined as the target fitting plane.

3. The method according to claim 2, characterized in that When the step of performing plane fitting using the data of multiple points in the historical point cloud to obtain the fitted plane in the target coordinate system is performed for the Nth time, the data of the multiple points used for plane fitting are the data of multiple inner points among the multiple points used for the N-1th plane fitting, where N is a positive integer and N≥2.

4. The method according to claim 1, wherein The data of the historical point cloud and the data of the current point cloud are data obtained by the depth sensor in the mobile device, and the target coordinate system is a coordinate system constructed with the target point in the mobile device as the origin, wherein the target point is a point that does not belong to the depth sensor.

5. The method according to claim 4, characterized in that The data of the historical point cloud is P k ={P k0 ,P k1 ,...,P kn }, where P k is the data in the depth sensor coordinate system, and the posture of the mobile device corresponding to each point in the historical point cloud is T k ={T k0 ,T k1 ,...,T kn },in, The performing plane fitting using the data of the historical point cloud to obtain a target fitting plane in a target coordinate system includes: Using the formula P k Converted into historical point cloud data in the target coordinate system, where is the historical point cloud data in the target coordinate system, T curr is the posture of the mobile device at the current moment, An external parameter for converting the depth sensor coordinate system to the target coordinate system; Using the historical point cloud data in the target coordinate system Performing plane fitting to obtain the target fitting plane in the target coordinate system; If the data of the current point cloud is not data in the target coordinate system, converting the data of the current point cloud into data in the target coordinate system includes: If the data of the current point cloud is the data in the depth sensor coordinate system, then use the formula The data of the current point cloud in the depth sensor coordinate system is converted into data in the target coordinate system, wherein: is the data of the current point cloud in the target coordinate system, P curr The data of the current point cloud in the depth sensor coordinate system is obtained by the depth sensor.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining an obstacle point cloud within a second historical period during the movement of the mobile device, wherein the obstacle point cloud is a collection of obstacle points at each moment within the second historical period; Each obstacle point in the obstacle point cloud within the second historical period is determined as the obstacle point at the current moment.

7. The method according to claim 6, characterized in that The data of the obstacle point cloud in the second historical period is P k '={P k0 ',P k1 ',...,P kn '}, the P k ' is the data in the target coordinate system, and the posture of the mobile device corresponding to each point in the obstacle point cloud in the second historical period is T k '={T k0 ',T k1 ',...,T kn '},in, The determining each obstacle point in the obstacle point cloud within the second historical period as the obstacle point at the current moment includes: Using the formula The obstacle point cloud data in the second historical period is converted into the point cloud data at the current moment, wherein: is the data of the obstacle point cloud at the current moment in the converted target coordinate system, T curr The posture of the mobile device at the current moment; Will The obstacle point in is determined as the obstacle point at the current moment.

8. The method according to claim 1, characterized in that The mobile device includes a depth sensor, and the historical point cloud data and the current point cloud data are both coordinate data of detection points within a detection blind area of a main sensor of the mobile device obtained by the depth sensor.

9. An obstacle point detection device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store executable instructions, and the executable instructions enable the processor to execute the obstacle detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores executable instructions, and the executable instructions execute the obstacle point detection method according to any one of claims 1 to 7 when running.

Images