Obstacle detection method, device, equipment and storage medium

Abstract

The present application belongs to the field of intelligent driving technology, and provides an obstacle detection method, device, equipment and storage medium, including: obtaining the original point cloud and the acquisition time of the original point cloud, the original point cloud is obtained through at least two different viewpoints; according to the acquisition time, the original point cloud is time-synchronized to obtain the point cloud at the same time; according to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same time is converted to a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system; the intermediate point cloud is merged to obtain a merged point cloud; the merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to the coordinate information of the processed point cloud. The present application can detect and identify obstacles with high precision and high efficiency, and meet the needs of high-precision obstacle identification in various environments.

Description

Technical Field

[0001] The present application belongs to the field of intelligent driving technology, and specifically relates to an obstacle detection method, device, equipment and storage medium. Background Art

[0002] The concept of obstacle detection is widely present in many fields such as intelligent driving and intelligent transportation. It mainly refers to the detection and identification of obstacles encountered by automobiles and other transportation vehicles during driving through a combination of software and hardware, so as to obtain information such as the type and size of the obstacle. Obstacle detection is of great significance for avoiding the impact of obstacles on driving and operating processes, reducing accident risks, and improving driving quality. It is also an indispensable technical support for technologies such as intelligent driving and autonomous driving.

[0003] At present, the most common obstacle detection method is to collect information on the driving path through devices such as cameras, sensors or radars, and then identify the collected information through a controller to determine whether there is an obstacle. For example, the patent publication number CN108627844A discloses an obstacle detection method, including collecting information through an ultrasonic sensor or a radar sensor, and identifying the information through an obstacle processing unit, so as to determine the obstacle. The disadvantage of the existing technology is that the accuracy of obstacle detection and identification is often poor, the efficiency of processing the collected information is low, and it is difficult to meet the obstacle identification requirements under high-precision requirements.

[0004] In summary, seeking a high-precision and high-efficiency obstacle detection method to meet the needs of high-precision obstacle recognition in various environments is of unparalleled importance in the field of intelligent driving technology. Summary of the invention

[0005] The present application provides an obstacle detection method, device, equipment and storage medium, which are used to solve the problem that the obstacle detection method in the prior art has poor accuracy in detecting and identifying obstacles, low efficiency in processing collected information, and difficulty in meeting the obstacle identification needs under high-precision requirements.

[0006] In a first aspect, an obstacle detection method is provided, comprising:

[0007] Acquire an original point cloud and a collection time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints;

[0008] According to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same time;

[0009] According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same time is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system;

[0010] Merging the intermediate point clouds to obtain a merged point cloud;

[0011] The merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to coordinate information of the processed point cloud.

[0012] Furthermore, the performing time synchronization processing on the original point cloud according to the acquisition time to obtain the point cloud at the same time includes:

[0013] According to the acquisition time and the system time of the original point cloud, a time difference between the acquisition time and the system time is obtained;

[0014] The original point cloud is compensated based on the time difference to synchronize the time of the original point cloud and obtain the point cloud at the same moment.

[0015] Furthermore, the converting the original point cloud to the vehicle coordinate system according to the conversion relationship between each viewpoint and the vehicle coordinate system to obtain an intermediate point cloud includes:

[0016] Obtaining a rotation and translation matrix representing the transformation relationship according to calibration parameters of the viewpoint relative to the vehicle coordinate system;

[0017] The original point cloud is calculated according to the rotation and translation matrix to obtain an intermediate point cloud.

[0018] Furthermore, after merging the intermediate point clouds to obtain a merged point cloud, the process includes:

[0019] Selecting an area of ​​interest for the merged point cloud according to the driving path information;

[0020] According to the selected region of interest, the portion of the merged point cloud that falls outside the region of interest is screened out, and the portion of the merged point cloud that falls within the region of interest is retained.

[0021] Furthermore, after merging the intermediate point clouds to obtain a merged point cloud, the process includes:

[0022] Identifying points representing ground in the merged point cloud according to coordinate information of points in the merged point cloud;

[0023] Points representing the ground in the merged point cloud are filtered out according to the recognition result.

[0024] Furthermore, clustering the merged point cloud to obtain a processed point cloud includes:

[0025] Performing preliminary clustering on the merged point cloud from a bird's-eye view perspective to obtain a preliminary clustered point cloud;

[0026] The initially clustered point cloud is formally clustered in the front view perspective to obtain a processed point cloud.

[0027] Furthermore, after initially clustering the merged point cloud from a bird's-eye view perspective, the method further includes:

[0028] Identifying noise points in the initial clustered point cloud according to coordinate information of points in the initial clustered point cloud;

[0029] The noise points in the initial clustered point cloud are screened out according to the recognition result.

[0030] Furthermore, the obstacle information is obtained according to the coordinate information of the processed point cloud, including

[0031] According to a pre-selected fitting direction, performing fitting processing on the processed point cloud in the fitting direction;

[0032] According to the result of the fitting process, a convex hull frame is generated, and the convex hull frame includes the obstacle information.

[0033] In a second aspect, an obstacle detection device is provided, comprising:

[0034] A data acquisition unit, used to acquire an original point cloud and an acquisition time of the original point cloud, wherein the original point cloud is obtained from at least two different viewpoints;

[0035] A time synchronization module, used to perform time synchronization processing on the original point cloud according to the acquisition time to obtain a point cloud at the same time;

[0036] A coordinate conversion module, used for the conversion relationship between the viewpoint and the vehicle coordinate system, performs a coordinate system conversion on the point cloud at the same time to obtain an intermediate point cloud in the vehicle coordinate system;

[0037] A splicing module, used for merging the intermediate point clouds to obtain a merged point cloud;

[0038] A clustering module, used for performing clustering processing on the merged point cloud to obtain a processed point cloud;

[0039] The post-processing module is used to obtain obstacle information based on the coordinate information of the processed point cloud.

[0040] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned obstacle detection method when executing the computer program.

[0041] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and wherein the computer program implements the steps of the above-mentioned obstacle detection method when executed by a processor.

[0042] In the scheme implemented by the above-mentioned obstacle detection method, device, equipment and storage medium, when facing the need for accurate obstacle detection, by obtaining the original point cloud and the acquisition time of the original point cloud, the original point cloud is obtained through at least two different viewpoints, and according to the acquisition time, the original point cloud is time-synchronized to obtain the point cloud at the same moment, and according to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same moment is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system, the intermediate point cloud is merged to obtain a merged point cloud, and the merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to the coordinate information of the processed point cloud, so as to maintain high-precision and high-efficiency detection and identification of obstacles, and meet the needs of high-precision obstacle identification in various environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of an application environment of an obstacle detection method in an exemplary embodiment of the present application;

[0044] Figure 2 is a process of an obstacle detection method in an exemplary embodiment of the present application;

[0045] Figure 3 yes Figure 2 A schematic flow chart of an exemplary specific implementation of step S220;

[0046] Figure 4 yes Figure 2 A schematic flow chart of an exemplary specific implementation of step S230;

[0047] Figure 5 yes Figure 2 An exemplary specific implementation of step S240 may include a flowchart of the steps;

[0048] Figure 6 yes Figure 2 Another exemplary specific implementation of step S240 may include a flowchart of the steps;

[0049] Figure 7 yes Figure 2 A schematic flow chart of an exemplary specific implementation of step S250;

[0050] Figure 8 yes Figure 7An exemplary specific implementation of step S710 may include a flowchart of the steps;

[0051] Fig. 9 yes Figure 2 Another exemplary specific implementation flow diagram of step S250;

[0052] Fig.10 is a structural schematic diagram of an obstacle detection device shown in an exemplary embodiment of the present application;

[0053] Fig.11 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0054] Among them, 101 is a car; 102 is a collection terminal; 103 is a control terminal; 104 is an object to be detected;

[0055] 1001-data acquisition unit; 1002-time synchronization module; 1003-coordinate conversion module; 1004-stitching module; 1005-clustering module; 1006-post-processing module; 1007-region of interest selection module; 1008-ground filtering module. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] The present application provides an obstacle detection method that can be applied to the detection of obstacles when a vehicle is traveling in various application scenarios. In one embodiment, it can be applied in, for example, Figure 1 In the application environment, Figure 1This is a schematic diagram of an application environment of an obstacle detection method in an embodiment of the present application, wherein the acquisition terminal 102 and the control terminal 103 are both arranged on the automobile 101. When the automobile 101 is driving, the acquisition terminal 102 arranged on the automobile 101 collects information of the object to be detected 104, and the control terminal 103 processes the information of the object to be detected 104 to identify the object to be detected 104, wherein the acquisition terminal 102 and the control terminal 103 can communicate through, for example, an Ethernet connection. From at least two viewpoints of the acquisition terminal 102, the control terminal 103 obtains at least two original point clouds of the object to be detected 104, and according to the acquisition time, the original point clouds are time-synchronized to obtain the point clouds at the same time, and according to the conversion relationship between the viewpoint and the vehicle coordinate system, the point clouds at the same time are converted to the coordinate system to obtain the intermediate point cloud in the vehicle coordinate system, and the intermediate point clouds are merged to obtain the merged point cloud. In the present application, the original point cloud of the object to be detected 104 and the acquisition time of the original point cloud are obtained from at least two viewpoints of the acquisition end 102, and the size information of the object to be detected 104 is obtained by processing the acquired original point cloud and the acquisition time through the control end 103. In the above application scenario, in view of the precise detection requirements of obstacles, the original point cloud and the acquisition time of the original point cloud are obtained, and the original point cloud is obtained through at least two different viewpoints. According to the acquisition time, the original point cloud is time-synchronized to obtain the point cloud at the same moment at the same time. According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same moment is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system. The intermediate point cloud is merged to obtain a merged point cloud. The merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to the coordinate information of the processed point cloud. It is possible to detect and identify obstacles with high precision and high efficiency, and meet the requirements for high-precision obstacle identification in various environments.

[0058] In one embodiment, see Figure 2 , Figure 2 The following is a flowchart of an obstacle detection method in an embodiment of the present application, comprising the following steps:

[0059] Step S210: Acquire an original point cloud and a collection time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints.

[0060] The viewpoint here generally refers to the location of the collection end 102 for collecting information about obstacles. The collection end 102 includes but is not limited to: cameras, millimeter wave radars, sonic radars, and lidars. The number of viewpoints is at least two. The information about obstacles is collected from each viewpoint from the perspective of the viewpoint, so that the same number of original point clouds as the number of viewpoints can be obtained at the same time. The original point cloud here refers to the set of points representing the obstacles obtained by collecting information about obstacles from each viewpoint.

[0061] In some embodiments, each viewpoint may be provided with a laser radar, for example, and the number of viewpoints may be two, for example, and the devices for collecting information about obstacles at each viewpoint are respectively the first laser radar and the second laser radar, and the first laser radar and the second laser radar collect information about the same obstacle from different viewpoints and different viewing angles, respectively, and obtain the first original point cloud and the second original point cloud, respectively, and the acquisition time of the point cloud is respectively the first acquisition time when the first original point cloud is collected and the second acquisition time when the second original point cloud is collected. It should be understood that the first original point cloud and the second original point cloud can both represent the characteristics of the surface contour of the same obstacle at the same time, but due to the errors in the calibration and acquisition process of the first laser radar and the second laser radar, the first original point cloud and the second original point cloud do not correspond in time, that is, the first original point cloud and the second original point cloud obtained at the same time have different corresponding first acquisition time and second acquisition time. It can be understood that the acquisition time here generally refers to the time when the obstacle detection starts from the viewpoint, and the acquisition method can be, for example, reading the timestamp of the time when the obstacle detection starts from the viewpoint.

[0062] Step S220: performing time synchronization processing on the original point cloud according to the acquisition time to obtain a point cloud at the same time.

[0063] After acquiring multiple original point clouds, in order to eliminate the temporal mismatch between each original point cloud, it is necessary to synchronize the multiple original point clouds according to the acquisition time to obtain the simultaneous point clouds at the same time.

[0064] In one embodiment, if Figure 3 As shown, Figure 3 yes Figure 2 FIG. 1 is a flow chart of an exemplary specific implementation of step S220 in FIG. 1 . In step S220, that is, according to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same time, and the following steps are also included:

[0065] Step S310: According to the acquisition time and the system time of the original point cloud, the time difference between the acquisition time and the system time is obtained.

[0066] Step S320: Compensating the original point cloud based on the time difference to synchronize the time of the original point cloud and obtain the point cloud at the same moment.

[0067] For steps S310-S320, when the original point cloud is time-synchronized, it is first necessary to obtain the time difference between the acquisition time and the system time based on the acquired acquisition time and the system time of the original point cloud. It should be understood that each time difference obtained here represents the time difference between the acquisition time and the system time of an original point cloud. After obtaining the time difference, the original point cloud is compensated based on the time difference so that multiple original point clouds complete time synchronization and obtain the point cloud at the same time. It should be understood that the compensation processing here generally refers to the original point cloud corresponding to the acquisition time as the calculation basis, and the change of the original point cloud from the acquisition time to the system time is calculated based on the obtained time difference, so as to obtain the point cloud at the same time corresponding to the system time. The calculation here can be achieved by means including but not limited to the Kalman filter algorithm. It can be understood that the system time generally refers to the time when the control system, for example, the control terminal 3, obtains the original point cloud, and the method of obtaining it can be, for example, reading the timestamp when the original point cloud is obtained.

[0068] Specifically, for example, in some embodiments, the first laser radar obtains the acquisition time T1 of the first original point cloud, and the second laser radar obtains the acquisition time T2 of the second original point cloud. In combination with the controller 3, the system time Ts of the first original point cloud and the second original point cloud is obtained to obtain a first time difference Ts-T1 corresponding to the first original point cloud and a second time difference Ts-T2 corresponding to the second original point cloud. According to the first time difference Ts-T1, the Kalman filter algorithm is used to calculate the change of the first original point cloud caused by the movement of the vehicle within the first time difference Ts-T1, and the first original point cloud is converted into the first time difference Ts-T2. The changes within the time difference Ts-T1 are compensated to the first original point cloud, so that the first original point cloud is synchronized to the system time, and the first simultaneous point cloud is obtained. According to the second time difference Ts-T2, the Kalman filter algorithm is used to calculate the changes of the second original point cloud caused by the vehicle movement within the second time difference Ts-T2, and the changes of the second original point cloud within the second time difference Ts-T2 are compensated to the second original point cloud, so that the second original point cloud is synchronized to the system time, and the second simultaneous point cloud is obtained. The first simultaneous point cloud and the second simultaneous point cloud obtained by the above implementation method both use the system time Ts as the time reference.

[0069] In the above steps, the time difference between the acquisition time and the system time is obtained according to the acquisition time and the system time of the original point cloud, and then the original point cloud is compensated based on the time difference to synchronize the time of the original point cloud, so as to obtain the point cloud at the same moment representing the obstacle at the same moment, which is conducive to eliminating errors in the calibration and acquisition process and improving the accuracy of obstacle detection.

[0070] Step S30: performing a coordinate system transformation on the point cloud at the same time according to the transformation relationship between the viewpoint and the vehicle coordinate system to obtain an intermediate point cloud in the vehicle coordinate system.

[0071] After the original point cloud completes time synchronization to obtain the simultaneous point cloud, it is necessary to convert the simultaneous point cloud to the vehicle coordinate system based on the transformation relationship between the viewpoint and the vehicle coordinate system, so that the simultaneous point clouds have the same spatial reference benchmark, so that the obstacles represented by the obtained multiple simultaneous point clouds are based on the same spatial reference benchmark and thus associated.

[0072] In one embodiment, if Figure 4 As shown, Figure 4 yes Figure 2 FIG. 1 is a flow chart of an exemplary specific implementation of step S230 in FIG. 1 . In step S230, that is, according to the conversion relationship between each viewpoint and the vehicle coordinate system, the original point cloud is converted to the vehicle coordinate system to obtain an intermediate point cloud, including the following steps:

[0073] Step S410: obtaining a rotation and translation matrix representing the conversion relationship according to calibration parameters of the viewpoint relative to the vehicle coordinate system;

[0074] Step S420: Calculate the original point cloud according to the rotation and translation matrix to obtain an intermediate point cloud.

[0075] For steps S410-S420, it is first necessary to obtain a rotation and translation matrix that characterizes the conversion relationship from the viewpoint to the vehicle coordinate system based on the calibration parameters of the viewpoint relative to the vehicle coordinate system. The calibration parameters here generally refer to the coordinate parameters of the viewpoint position in the vehicle coordinate system. It should be understood that the conversion relationship can be a conversion method represented by parameters including but not limited to three-dimensional rectangular coordinates, three-dimensional angles, and six-axis postures. The rotation and translation matrix generally refers to a matrix that is obtained through the conversion relationship and can characterize the conversion relationship. The original point cloud is calculated through the rotation and translation matrix, and the original point cloud is converted to the vehicle coordinate system to obtain an intermediate point cloud with the vehicle coordinate system as the reference coordinate.

[0076] Specifically, for example, in some embodiments, the first laser radar collects a first original point cloud, and the second laser radar collects a second original point cloud. The calibration parameters of the first laser radar are first calibration parameters (x1, y1, z1, roll1, pitch1, yaw1), and the first calibration parameters characterize the position and angle of the viewpoint of the first laser radar relative to the vehicle coordinate system. The calibration parameters of the second laser radar are second calibration parameters (x2, y2, z2, roll2, pitch2, yaw2), and the second calibration parameters characterize the position and angle of the viewpoint of the second laser radar relative to the vehicle coordinate system. The first rotation and translation matrix and the second rotation and translation matrix are obtained by the first calibration parameters and the second calibration parameters. The first original point cloud is calculated by the first rotation and translation matrix, and the first original point cloud is converted into a first intermediate point cloud in the vehicle coordinate system. The second original point cloud is calculated by the second rotation and translation matrix, and the second original point cloud is converted into a second intermediate point cloud in the vehicle coordinate system.

[0077] Step S240: merging the intermediate point clouds to obtain a merged point cloud.

[0078] After obtaining multiple intermediate point clouds representing obstacles and based on the same spatial reference benchmark, the intermediate point clouds need to be merged to obtain a merged point cloud in the vehicle coordinate system. It can be understood that the merged point cloud here includes the information in each intermediate point cloud. Specifically, for example, in some embodiments, a first intermediate point cloud and a second intermediate point cloud are obtained. The method for merging the first intermediate point cloud and the second intermediate point cloud can be, for example, that there are 2000 points in the first intermediate point cloud and 2000 points in the second intermediate point cloud. According to the coordinates of 4000 points in the first intermediate point cloud and the second intermediate point cloud, corresponding 4000 points are generated in the same point cloud to obtain an intermediate point cloud. It should be understood that the 2000 points in the first intermediate point cloud and the 2000 points in the second intermediate point cloud may be points with repeated coordinates. For example, in some embodiments, there are 200 points with completely repeated coordinates among the 2000 points in the first intermediate point cloud and the 2000 points in the second intermediate point cloud. In the generated intermediate point cloud, there are a total of 3800 points with non-repeating coordinates and 200 points with completely repeated coordinates with the other 3800 points.

[0079] Through the above step S240, the acquired data is merged into a merged point cloud, which is conducive to reducing errors in detection and improving obstacle detection accuracy and detection efficiency.

[0080] In one embodiment, if Figure 5 As shown, Figure 5 yes Figure 2An exemplary specific implementation of step S240 may include a flowchart of steps. After step S240, that is, after the intermediate point clouds are merged to obtain the merged point cloud, the following steps may be included:

[0081] Step S510: selecting an area of ​​interest for the merged point cloud according to the driving path information.

[0082] Step S520: According to the selected region of interest, the portion of the merged point cloud that falls outside the region of interest is screened out, and the portion of the merged point cloud that falls within the region of interest is retained.

[0083] For steps S510-S520, it is first necessary to select an area of ​​interest for the merged point cloud based on the driving path information. Specifically, in some embodiments, the method of obtaining the driving path information includes but is not limited to obtaining it based on a map provided by the Global Positioning System (GPS), pre-collecting and mapping the driving path, and real-time identification of the driving path. Based on the acquired driving path information, an area of ​​interest is selected, and based on the selected area of ​​interest, points in the merged point cloud whose coordinates fall within the boundary of the area of ​​interest are retained, and points in the merged point cloud whose coordinates fall outside the boundary of the area of ​​interest are screened out.

[0084] Specifically, for example, in some embodiments, the width of the driving path can be obtained by, for example, real-time identification of the driving path through a radar to obtain the width of the driving path. It can be understood that the driving path width here refers to the driving path width corresponding to the system time when the original point cloud is acquired. The area within the driving path width is set as the area of ​​interest. According to the set area of ​​interest, all points in the merged point cloud are screened, points falling outside the area of ​​interest are screened out, and points falling within the area of ​​interest are retained. In the merged point cloud after screening, only points in the area within the driving path width are retained.

[0085] Through the above steps S510-S520, redundant data in the merged point cloud is filtered out, which is beneficial to reducing the amount of data to be processed and improving the efficiency of obstacle detection.

[0086] In one embodiment, if Figure 6 As shown, Figure 6 yes Figure 2 Another exemplary specific implementation of step S240 may include a flowchart of steps. After step S240, that is, after the intermediate point clouds are merged to obtain the merged point cloud, the following steps may be included:

[0087] Step S610: identifying points representing the ground in the merged point cloud according to the coordinate information of the points in the merged point cloud.

[0088] Step S620: filtering out points representing the ground in the merged point cloud according to the recognition result.

[0089] For steps S610-S620, it is necessary to identify the points representing the ground in the merged point cloud based on the coordinate information of the points in the merged point cloud. The method for identifying the points representing the ground based on the coordinate information here includes but is not limited to a plane fitting algorithm, a random sampling consensus algorithm, etc. The points representing the ground in the merged point cloud are filtered out based on the recognition result.

[0090] Specifically, for example, in some embodiments, based on the coordinate information of the points in the merged point cloud that have been acquired, a random sampling consensus algorithm is used to calculate the points in the merged point cloud to identify the points in the merged point cloud that represent the ground. Based on the identification results, the points in the merged point cloud that represent the ground are filtered out, and only points that represent obstacles are retained.

[0091] Through the above steps S610-S620, the data of the points representing the ground in the merged point cloud are filtered out, which is beneficial to reducing the amount of data to be processed and improving the efficiency of obstacle detection.

[0092] Step S250: clustering the merged point cloud to obtain a processed point cloud, so as to obtain obstacle information according to coordinate information of the processed point cloud.

[0093] In step S250, the merged point cloud needs to be clustered. It should be understood that clustering the merged point cloud generally refers to dividing the points in the point cloud into clusters according to the clustering conditions between the points to obtain a processed point cloud. Each point cloud cluster in the processed point cloud represents information about, for example, an obstacle. It can be understood that the specific method of clustering processing may be, for example, calculating the merged point cloud through a clustering algorithm. The principle of the clustering algorithm here includes but is not limited to determining whether points belong to the same cluster based on the number of other points in the neighborhood of each point in the merged point cloud, or determining whether points belong to the same cluster based on the number of connected points of each point. Specifically, for example, in some embodiments, the method of clustering the merged point cloud is to calculate the number of other points in the neighborhood of each point in the merged point cloud based on a neighborhood range set in advance, and determine whether the point and the points in the neighborhood belong to the same cluster based on the number of other points in the neighborhood. In other embodiments, the method of clustering the merged point cloud is to calculate the number of connected points of each point according to the definition range of connected points set in advance, and determine whether the point and its connected points belong to the cluster according to the number of connected points of the point. After obtaining the processed point cloud, the obstacle information is obtained according to the coordinate information of the processed point cloud. It can be understood that the obstacle information here includes but is not limited to the obstacle shape directly displayed by the processed point cloud, the coordinates and size parameters calculated by the coordinates of the points in the processed point cloud, such as the maximum length of the obstacle, the maximum width of the obstacle, the maximum height of the obstacle, etc. Specifically, for example, in some embodiments, there are 300 points in a cluster of the post-processing point cloud, and the maximum height coordinate zmax and the minimum height coordinate zmin of the 300 points are read, and zmax-zmin is calculated to obtain the maximum height of the obstacle.

[0094] In one embodiment, if Figure 7 As shown, Figure 7 yes Figure 2 FIG. 1 is a flow chart of an exemplary specific implementation of step S250 in FIG. 1 . In step S250, clustering is performed on the merged point cloud to obtain a processed point cloud, which specifically includes the following steps:

[0095] Step S710: performing preliminary clustering on the merged point cloud from a bird's-eye view perspective to obtain a preliminary clustered point cloud.

[0096] Step S720: Formal clustering is performed on the initially clustered point cloud in the front view perspective to obtain a processed point cloud.

[0097] For steps S710-S720, it is first necessary to perform preliminary clustering on the merged point cloud from a bird's-eye view perspective to obtain a preliminary clustered point cloud, and then perform formal clustering on the preliminary clustered point cloud from a front view perspective to obtain a processed point cloud. It should be understood that each point in the merged point cloud contains the coordinate information of the point, but the density of the point cloud distribution of the merged point cloud in the bird's-eye view perspective is less than the density of the points in the point cloud in the front view perspective. Clustering is performed in different directions, and the accuracy of the clustering processing results is also affected by the point density of the point cloud. Performing preliminary clustering from a bird's-eye view perspective and then formal clustering from a front view perspective is beneficial to improving the accuracy of clustering processing, thereby improving the accuracy of obstacle detection.

[0098] In one embodiment, if Figure 8 As shown, Figure 8 yes Figure 7 An exemplary specific implementation of step S710 may include a flowchart of steps, after step S710, that is, after the merged point cloud is initially clustered from a bird's-eye view perspective to obtain the initial clustered point cloud, the following steps are also included:

[0099] Step S810: Identifying noise points in the initial clustered point cloud according to the coordinate information of the points in the initial clustered point cloud.

[0100] Step S820: Screening out the noise points in the initial clustered point cloud according to the recognition result.

[0101] For steps S810-S820, after the merged point cloud is initially clustered from a bird's-eye view perspective, noise points in the initially clustered point cloud are identified based on the coordinate information of the points in the initially clustered point cloud. It should be understood that the noise points here generally refer to points that are irrelevant to the detection target and appear in the point cloud due to the influence of influencing factors. The influencing factors here include but are not limited to floating impurities, light, pulse interference, and interference. The method of identifying noise points through the coordinate information of points in the initially clustered point cloud includes but is not limited to calculating the number of other points in the neighborhood of each point in the initially clustered point cloud to determine whether the point is a noise point, or calculating the number of connected points of each point to determine whether the point is a noise point. According to the identification results of the noise points in the initially clustered point cloud, the noise points are screened out.

[0102] Specifically, for example, in some embodiments, noise points are identified based on the coordinate information of points in the initial clustered point cloud. The noise point identification method is, for example, based on a pre-set connection point judgment standard, for example, if the interval distance is not greater than 1 mm, it is determined to be a connected point, and the number of connected points of each point is calculated. If the number of connected points is 0, the point is judged to be a noise point. Each point in the initial clustered point cloud is calculated, and the noise points in the initial clustered point cloud are identified. According to the recognition result, the noise points in the initial clustered point cloud are screened out.

[0103] Through the above steps S810-S820, the noise points in the initial clustered point cloud are screened out, thereby eliminating the interference of the noise points on obstacle detection, which is conducive to improving the efficiency of obstacle detection.

[0104] In one embodiment, if Fig. 9 As shown, Fig. 9 yes Figure 2 Another exemplary specific implementation flow chart of step S250 in step S250, in which the obstacle information is obtained according to the coordinate information of the processed point cloud, the following steps are specifically included:

[0105] Step S910: According to a pre-selected fitting direction, fitting processing is performed on the processed point cloud in the fitting direction.

[0106] Step S920: generating a convex hull frame according to the result of the fitting process, wherein the convex hull frame includes the obstacle information.

[0107] For steps S910-S920, first, the processed point cloud needs to be fitted in the pre-selected fitting direction. It can be understood that the fitting direction here can be, for example, the same one or more directions can be selected for all clusters in the processed point cloud, or different one or more directions can be selected for each cluster, depending on the actual needs of obstacle identification. The fitting process here generally refers to enclosing the points in a cluster in the fitting direction using geometric representation methods including but not limited to closed boxes and closed curves to characterize the shape of the obstacle. After the processed point cloud is fitted, a convex hull frame is generated according to the result of the fitting process. The convex hull frame contains the obstacle information. It can also be understood that the obstacle information represented by the convex hull frame can be intuitively obtained through the convex hull frame.

[0108] Specifically, for example, in some embodiments, the roll angle direction is selected as the fitting direction of the obstacle. It can be understood that the roll angle direction generally refers to the direction of rotation around the driving direction. In the roll angle direction, all points of a cluster in the processed point cloud are fitted by a convex hull algorithm to obtain the maximum outer shape boundary that can characterize the obstacle in the roll angle direction. The outer shape boundary is, for example, an irregular hexadecagon. Based on the irregular hexadecagon obtained by fitting, a convex hull frame that just includes all points of the cluster is generated. The convex hull frame is, for example, an irregular hexadecagon.

[0109] Through the above steps S910-S920, specific information of the obstacle is obtained according to the processed point cloud, including fitting the processed point cloud in the pre-selected fitting direction, and generating a convex hull frame containing the obstacle information according to the result of the fitting process, so that the obstacle information is represented in the form of a convex hull frame, which can ensure accurate detection and identification of the obstacle information, and is conducive to further improving the accuracy of obstacle detection.

[0110] It can be seen that in the above scheme, in order to meet the demand for accurate detection of obstacles, the original point cloud and the acquisition time of the original point cloud are obtained, and the original point cloud is obtained through at least two different viewpoints. According to the acquisition time, the original point cloud is time-synchronized to obtain the point cloud at the same moment, and according to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same moment is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system. The intermediate point cloud is merged to obtain a merged point cloud, and the merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to the coordinate information of the processed point cloud, so as to maintain high-precision and high-efficiency detection and identification of obstacles, and meet the needs of high-precision obstacle identification in various environments.

[0111] In one embodiment, an obstacle detection device is provided, which corresponds one-to-one to the obstacle detection method in the above embodiment, such as Fig.10 As shown, Fig.10 100 is a schematic diagram of a structure of an obstacle detection device shown in an exemplary embodiment of the present application, including a data acquisition unit 1001, a time synchronization module 1002, a coordinate conversion module 1003, a splicing module 1004, a clustering module 1005 and a post-processing module 1006. The functional modules are described in detail as follows:

[0112] The data acquisition unit 1001 is used to acquire an original point cloud and the acquisition time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints.

[0113] The time synchronization module 1002 is used to perform time synchronization processing on the original point cloud according to the acquisition time to obtain the point cloud at the same time.

[0114] The coordinate conversion module 1003 is used for the conversion relationship between the viewpoint and the vehicle coordinate system, and performs coordinate system conversion on the point cloud at the same time to obtain an intermediate point cloud in the vehicle coordinate system.

[0115] The stitching module 1004 is used to merge the intermediate point clouds to obtain a merged point cloud.

[0116] The clustering module 1005 is used to perform clustering processing on the merged point cloud to obtain a processed point cloud.

[0117] The post-processing module 1006 is used to obtain obstacle information according to the coordinate information of the processed point cloud.

[0118] In some implementations, the time synchronization module 1002 is specifically configured to:

[0119] According to the acquisition time and the system time of the original point cloud, a time difference between the acquisition time and the system time is obtained;

[0120] The time difference is compensated to the original point cloud to synchronize the time of the original point cloud and obtain the point cloud at the same moment.

[0121] In some implementations, the coordinate conversion module 1003 is specifically used to:

[0122] Obtaining a rotation and translation matrix representing the transformation relationship according to calibration parameters of the viewpoint relative to the vehicle coordinate system;

[0123] The original point cloud is calculated according to the rotation and translation matrix to obtain an intermediate point cloud.

[0124] In some implementations, the clustering module 1005 is specifically configured to:

[0125] Performing preliminary clustering on the merged point cloud from a bird's-eye view perspective to obtain a preliminary clustered point cloud;

[0126] The initially clustered point cloud is formally clustered in the front view perspective to obtain a processed point cloud.

[0127] In some implementations, the clustering module 1005 is further configured to:

[0128] Identifying noise points in the initial clustered point cloud according to coordinate information of points in the initial clustered point cloud;

[0129] The noise points in the initial clustered point cloud are screened out according to the recognition result.

[0130] In some implementations, the post-processing module 1006 is specifically configured to:

[0131] At least one fitting direction is selected, and a convex hull algorithm is used to perform fitting processing on the processed point cloud in the fitting direction.

[0132] According to the result of the fitting process, a convex hull frame representing the obstacle information is generated.

[0133] In some implementations, the obstacle detection device further includes an area of ​​interest selection module 1007, which is specifically used to:

[0134] After merging the intermediate point clouds to obtain a merged point cloud, the width of the driving path is obtained, and an area within the width of the driving path is selected as the area of ​​interest;

[0135] According to the selected region of interest, the portion of the merged point cloud that falls outside the region of interest is screened out, and the portion of the merged point cloud that falls within the region of interest is retained.

[0136] In some implementations, the obstacle detection device further includes a ground filtering module 1008, which is specifically configured to:

[0137] After merging the intermediate point clouds to obtain a merged point cloud, identifying points in the merged point cloud that represent the ground according to coordinate information of the points in the merged point cloud;

[0138] Points representing the ground in the merged point cloud are filtered out according to the recognition result.

[0139] The obstacle detection device provided in the present embodiment aims at the demand for accurate obstacle detection. By acquiring an original point cloud and the acquisition time of the original point cloud, the original point cloud is obtained through at least two different viewpoints. According to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same moment. According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same moment is subjected to coordinate system conversion to obtain an intermediate point cloud in the vehicle coordinate system. The intermediate point cloud is merged to obtain a merged point cloud. The merged point cloud is clustered to obtain a processed point cloud. Obstacle information is obtained according to the coordinate information of the processed point cloud. Obstacle detection and identification can be performed with high precision and high efficiency, meeting the demand for high-precision obstacle identification in various environments.

[0140] For the specific definition of an obstacle detection device, please refer to the definition of an obstacle detection method above, which will not be repeated here. Each module in the above obstacle detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer in hardware form, or can be stored in the memory of the computer in software form, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the computer here includes but is not limited to computers, car computers, detachable smart devices and other electronic devices.

[0141] In one embodiment, an obstacle detection device is provided. The device may be, for example, a detachable intelligent device including a collection terminal 102 and a control terminal 103, or may be, for example, a car 101 integrated with the collection terminal 102 and the control terminal 103. Specifically, a computer device, whose internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen, an input device and a collection device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to connect and communicate with the collection device, and is used to receive obstacle information obtained by the collection device from the viewpoint. When the computer program is executed by the processor, the following steps can be implemented:

[0142] Acquire an original point cloud and a collection time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints;

[0143] According to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same time;

[0144] According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same time is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system;

[0145] Merging the intermediate point clouds to obtain a merged point cloud;

[0146] The merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to coordinate information of the processed point cloud.

[0147] It should be understood that the network interface may include but is not limited to a local area network interface, a wide area network interface, and a wireless network interface, and may include a wireless network interface or a wired network interface, which is generally used to establish a communication connection between the computer device and other electronic devices. The network may be an intranet, the Internet, the Global System of Mobilecommunication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi Fi and other wireless or wired networks.

[0148] An obstacle detection device provided in an embodiment of the present application, in response to the demand for accurate obstacle detection, simultaneously acquires at least two original point clouds from at least two viewpoints; synchronizes the original point clouds at the acquisition time of each viewpoint by acquiring the original point clouds; converts the original point clouds to the vehicle coordinate system according to the transformation relationship of each viewpoint relative to the vehicle coordinate system to obtain an intermediate point cloud; merges the intermediate point clouds to obtain a merged point cloud; sets an area of ​​interest to filter the merged point cloud; uses a ground filtering algorithm to filter the merged point cloud; clusters the merged point cloud to obtain a processed point cloud; obtains the size information of the obstacle based on the processed point cloud, and is capable of detecting and identifying obstacles with high precision and efficiency, meeting the demand for high-precision obstacle identification in various environments.

[0149] In one embodiment, a device-readable storage medium is provided, on which a software program is stored, and when the software program is executed by a processor, the following steps are implemented:

[0150] Acquire an original point cloud and a collection time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints;

[0151] According to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same time;

[0152] According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same time is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system;

[0153] Merging the intermediate point clouds to obtain a merged point cloud;

[0154] The merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to coordinate information of the processed point cloud.

[0155] It should be noted that the equipment here includes but is not limited to computers, car computers, detachable smart devices and other electronic devices, and the software programs here include but are not limited to computer programs obtained by computer encoding.

[0156] It should be noted that the functions or steps that can be implemented by the device-readable storage medium or the device can be found in the aforementioned method embodiments. The relevant descriptions are not described one by one here to avoid repetition.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a software program, and the software program can be stored in a non-volatile device readable storage medium. When the software program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0158] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional unit modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional unit modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0159] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An obstacle detection method, characterized in that: include: Acquire an original point cloud and a collection time of the original point cloud, wherein the original point cloud is obtained through at least two different viewpoints; According to the acquisition time, the original point cloud is subjected to time synchronization processing to obtain a point cloud at the same time; According to the conversion relationship between the viewpoint and the vehicle coordinate system, the point cloud at the same time is converted into a coordinate system to obtain an intermediate point cloud in the vehicle coordinate system; Merging the intermediate point clouds to obtain a merged point cloud, including selecting an area of ​​interest for the merged point cloud according to the driving path information; filtering out a portion of the merged point cloud that falls outside the area of ​​interest according to the selected area of ​​interest, and retaining a portion of the merged point cloud that falls within the area of ​​interest; identifying points representing the ground in the merged point cloud according to coordinate information of points in the merged point cloud, and filtering out points representing the ground in the merged point cloud according to the identification result; The merged point cloud is clustered to obtain a processed point cloud, so as to obtain obstacle information according to coordinate information of the processed point cloud.

2. The obstacle detection method according to claim 1, characterized in that: The step of performing time synchronization processing on the original point cloud according to the acquisition time to obtain a point cloud at the same time includes: According to the acquisition time and the system time of the original point cloud, a time difference between the acquisition time and the system time is obtained; The original point cloud is compensated based on the time difference to synchronize the time of the original point cloud and obtain the point cloud at the same moment.

3. The obstacle detection method according to claim 1, characterized in that: The step of converting the original point cloud to the vehicle coordinate system according to the conversion relationship between each viewpoint and the vehicle coordinate system to obtain an intermediate point cloud includes: Obtaining a rotation and translation matrix representing the transformation relationship according to calibration parameters of the viewpoint relative to the vehicle coordinate system; The original point cloud is calculated according to the rotation and translation matrix to obtain an intermediate point cloud.

4. The obstacle detection method according to claim 1, characterized in that: The clustering process is performed on the merged point cloud to obtain a processed point cloud, including: Performing preliminary clustering on the merged point cloud from a bird's-eye view perspective to obtain a preliminary clustered point cloud; The initially clustered point cloud is formally clustered in the front view perspective to obtain a processed point cloud.

5. The obstacle detection method according to claim 4, characterized in that: After the initial clustering of the merged point cloud from a bird's eye view, the method further includes: Identifying noise points in the initial clustered point cloud according to coordinate information of points in the initial clustered point cloud; The noise points in the initial clustered point cloud are screened out according to the recognition result.

6. The obstacle detection method according to claim 1, characterized in that: Obtaining obstacle information according to the coordinate information of the processed point cloud includes: According to a pre-selected fitting direction, performing fitting processing on the processed point cloud in the fitting direction; According to the result of the fitting process, a convex hull frame is generated, and the convex hull frame includes the obstacle information.

7. An obstacle detection device, characterized in that: include: A data acquisition unit, used to acquire an original point cloud and an acquisition time of the original point cloud, wherein the original point cloud is obtained from at least two different viewpoints; A time synchronization module, used to perform time synchronization processing on the original point cloud according to the acquisition time to obtain a point cloud at the same time; A coordinate conversion module, used for the conversion relationship between the viewpoint and the vehicle coordinate system, performs a coordinate system conversion on the point cloud at the same time to obtain an intermediate point cloud in the vehicle coordinate system; a splicing module, used for merging the intermediate point clouds to obtain a merged point cloud, including selecting an area of ​​interest for the merged point cloud according to the driving path information; filtering out the part of the merged point cloud that falls outside the area of ​​interest according to the selected area of ​​interest, and retaining the part of the merged point cloud that falls within the area of ​​interest; identifying points representing the ground in the merged point cloud according to coordinate information of points in the merged point cloud, and filtering out points representing the ground in the merged point cloud according to the identification result; A clustering module, used for performing clustering processing on the merged point cloud to obtain a processed point cloud; The post-processing module is used to obtain obstacle information based on the coordinate information of the processed point cloud.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the obstacle detection method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the obstacle detection method according to any one of claims 1 to 6 are implemented.

Images