Obstacle detection method, device, equipment and storage medium

Abstract

The present invention relates to the field of radar detection technology, and discloses an obstacle detection method, device, equipment and storage medium. The obstacle detection method comprises: extracting voxel features from road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information; obtaining density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to adjacent voxels according to the multidimensional feature vectors; synthesizing the multidimensional feature vectors and the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to adjacent voxels to obtain target voxel features corresponding to each voxel; performing obstacle detection on the target voxel features through a voxel neural network model to obtain a road obstacle detection result, which does not significantly increase the overall calculation amount of the model, thereby improving the obstacle detection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of radar detection technology, and in particular to an obstacle detection method, device, equipment and storage medium. Background Art

[0002] In general, lidar perception solutions used in large-scale deployment of autonomous driving often use voxel-based neural network models. Voxel-based means voxelizing point clouds, and the basic network structure uses 3D convolution or 2D convolution.

[0003] However, the disadvantage of the voxel-based method is that it processes the point cloud too roughly. The general method is to set the voxel value to 0 or 1, that is, the voxel contains a point cloud if it is 1, otherwise it is 0. This approach causes the point cloud to lose too much information, resulting in low accuracy in obstacle detection by the neural network model. In order to reduce the information loss of the point cloud, the amount of calculation will also be doubled. Summary of the invention

[0004] The main purpose of the present invention is to provide an obstacle detection method, device, equipment and storage medium, aiming to solve the problem of how to improve the obstacle detection efficiency.

[0005] To achieve the above object, the present invention provides an obstacle detection method, which comprises the following steps:

[0006] Get road point cloud information;

[0007] Extracting voxel features from the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information;

[0008] The density-based voxel features and the reflectivity-based multi-dimensional feature vectors are used to detect obstacles through a voxel neural network model to obtain a road obstacle detection result.

[0009] Optionally, the performing obstacle detection on the density-based voxel features and the reflectivity-based multidimensional feature vectors through a voxel neural network model to obtain a road obstacle detection result includes:

[0010] According to the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels, the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels are obtained;

[0011] Synthesizing the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain target voxel features corresponding to each of the voxels;

[0012] The target voxel features are used to detect obstacles through a voxel neural network model to obtain a road obstacle detection result.

[0013] Optionally, the extracting voxel features from the road point cloud information to obtain density-based voxel features corresponding to each voxel in the road point cloud information includes:

[0014] Determine the target point cloud quantity and the actual point cloud quantity according to the road point cloud information;

[0015] Voxel features are extracted according to the target point cloud quantity and the actual point cloud quantity to obtain density-based voxel features corresponding to each voxel in the road point cloud information.

[0016] Optionally, determining the target point cloud quantity and the actual point cloud quantity according to the road point cloud information includes:

[0017] Dividing the three-dimensional space where the road point cloud information is located to obtain individual voxels;

[0018] Determining the point cloud information contained in each voxel according to the center point and range of each voxel;

[0019] The target point cloud quantity and the actual point cloud quantity are determined according to the point cloud information contained in the voxel.

[0020] Optionally, determining the target point cloud quantity and the actual point cloud quantity according to the point cloud information included in the voxel includes:

[0021] randomly selecting a current voxel from each of the voxels;

[0022] Determining point cloud information contained in the current voxel;

[0023] Counting the number of point clouds in the point cloud information contained in the current voxel;

[0024] The number of point clouds of the current voxel is used as the target point cloud number;

[0025] The number of point clouds in other voxels that is smaller than the target number of point clouds is taken as the actual number of point clouds.

[0026] Optionally, the extracting voxel features of the road point cloud information to obtain a reflectivity-based multidimensional feature vector corresponding to each voxel in the road point cloud information includes:

[0027] Extracting the reflectivity of the point cloud corresponding to each voxel in the road point cloud information;

[0028] A multidimensional feature vector is generated according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity.

[0029] Optionally, generating a multidimensional feature vector according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity includes:

[0030] Determine the number of point clouds corresponding to each of the voxels;

[0031] When the number of point clouds corresponding to each of the voxels reaches a target number, averaging the reflectivity of the point clouds corresponding to each of the voxels to obtain an average reflectivity;

[0032] A reflectivity-based multi-dimensional feature vector is generated according to the average reflectivity.

[0033] In addition, to achieve the above object, the present invention further provides an obstacle detection device, the obstacle detection device comprising:

[0034] Acquisition module, used to obtain road point cloud information;

[0035] An extraction module, used for performing voxel feature extraction on the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information;

[0036] The detection module is used to perform obstacle detection on the density-based voxel features and the reflectivity-based multi-dimensional feature vectors through a voxel neural network model to obtain a road obstacle detection result.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes an obstacle detection device, which includes: a memory, a processor, and an obstacle detection program stored in the memory and executable on the processor, wherein the obstacle detection program is configured to implement the obstacle detection method described above.

[0038] In addition, to achieve the above objective, the present invention further proposes a storage medium, on which an obstacle detection program is stored, and when the obstacle detection program is executed by a processor, the obstacle detection method as described above is implemented.

[0039] The obstacle detection method proposed by the present invention obtains road point cloud information; performs voxel feature extraction on the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information; obtains density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to adjacent voxels based on the density-based voxel features and reflectivity-based multidimensional feature vectors; synthesizes the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel and the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain target voxel features corresponding to each voxel; performs obstacle detection on the target voxel features through a voxel neural network model to obtain a road obstacle detection result. In order to further reduce the information loss of point clouds and avoid doubling the amount of calculation, the present invention provides a "3d spatial2channel" solution, which synthesizes multiple adjacent voxels into a "large voxel". Although the number of channels of the "large voxel" will increase, it only increases the amount of calculation of the first layer of the network, and does not significantly increase the overall calculation amount of the model, thereby improving the efficiency of obstacle detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the structure of an obstacle detection device in the hardware operating environment involved in the embodiment of the present invention;

[0041] Figure 2 It is a flowchart of a first embodiment of an obstacle detection method of the present invention;

[0042] Figure 3 A schematic diagram of road point cloud information of an embodiment of an obstacle detection method of the present invention;

[0043] Figure 4 This is an overall flow chart of an embodiment of an obstacle detection method of the present invention;

[0044] Figure 5 It is a flowchart of a second embodiment of an obstacle detection method of the present invention;

[0045] Figure 6 FIG. 1 is a schematic diagram of functional modules of the first embodiment of the obstacle detection device of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] Reference Figure 1 , Figure 1The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0049] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a button, and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1 The obstacle detection device structure shown in the figure does not constitute a limitation on the obstacle detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0051] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an obstacle detection program.

[0052] exist Figure 1 In the obstacle detection device shown, the network interface 1004 is mainly used to connect to the server and perform data communication with the server; the user interface 1003 is mainly used to connect to the user terminal and perform data communication with the terminal; the obstacle detection device of the present invention calls the obstacle detection program stored in the memory 1005 through the processor 1001, and executes the obstacle detection method provided by the embodiment of the present invention.

[0053] Based on the above hardware structure, an obstacle detection method embodiment of the present invention is proposed.

[0054] Reference Figure 2 , Figure 2 FIG. 1 is a flow chart of the first embodiment of the obstacle detection method of the present invention.

[0055] In a first embodiment, the obstacle detection method comprises the following steps:

[0056] Step S10, obtaining road point cloud information.

[0057] It should be noted that the execution subject of this embodiment is an obstacle detection device, which can be a vehicle. An obstacle detection program is provided on the vehicle, and obstacle detection can be performed according to the obstacle detection program. The vehicle includes a laser radar, which collects road point cloud information through the laser radar, and detects obstacles on the road according to the road point cloud information. The road point cloud information is a collection of laser radar scanning points. The laser radar system scans the ground to obtain the three-dimensional coordinates of the ground reflection point. Each ground reflection point is distributed in the three-dimensional space in the form of a point according to the three-dimensional coordinates, and a collection of laser radar scanning points is obtained, such as Figure 3 Schematic diagram of road point cloud information shown.

[0058] Step S20 , extracting voxel features from the road point cloud information to obtain density-based voxel features and reflectivity-based multi-dimensional feature vectors corresponding to each voxel in the road point cloud information.

[0059] In general, voxel is the smallest unit of digital data in three-dimensional space segmentation. When performing obstacle detection through a voxel neural network model, the voxel value is set to 0 or 1. If the voxel contains a point cloud, it is 1, otherwise it is 0. In this case, the point cloud is likely to lose more information. However, in this embodiment, the voxel value is set to a multidimensional vector of density and reflectivity to reduce the information loss of the point cloud, thereby improving the detection accuracy of the model, wherein the density-based voxel feature reflects the density of the point cloud in the voxel, and the reflectivity-based multidimensional feature vector reflects the reflectivity of the point cloud in the voxel.

[0060] like Figure 4 As shown in the overall flow chart, an embodiment is provided for specific explanation. The laser point cloud lidar is used as input, the size is (n, 4), n is the number of laser point clouds, and the 4 dimensions are x, y, z, and reflectivity. Among them, the value range of reflectivity is 0-255. Density_Reflectivity VFE, that is, a voxel feature extractor (Voxel Feature Extractor, VFE) based on density reflectivity, and Lidar 2D / 3D Model represents the processing voxel (H*W*Z*C) format, wherein H, W, Z and C represent the length, width, and height of the voxel and the feature vector dimension respectively. The voxel-based neural network model is updated in real time through the back propagation algorithm through the loss function, thereby improving the real-time performance of the voxel-based neural network model.

[0061] In a specific implementation, the extraction of density-based voxel features and reflectivity-based multi-dimensional feature vectors is achieved through a density-reflectivity-based voxel feature extractor. The density-reflectivity-based voxel feature extractor can be established in advance through set density calculation rules and reflectivity multi-dimensional vector generation rules, thereby realizing the generation of a density-reflectivity-based voxel feature extractor to achieve the density and reflectivity feature extraction of the point cloud.

[0062] The density-based voxel features and the reflectivity-based multi-dimensional feature vectors are used to detect obstacles through a voxel neural network model to obtain a road obstacle detection result.

[0063] It should be noted that the density-based voxel features can be represented by density, and the multidimensional feature vector based on reflectivity can be represented by reflectivity. The density-based voxel features and the multidimensional feature vector based on reflectivity are used for obstacle detection through a voxel neural network model, that is, density and reflectivity are combined to obtain the feature feature = [density, reflectivity]. Since obstacle detection is performed in combination with density and reflectivity, more attribute information of the point cloud is combined for detection, and more accurate road obstacle detection results can be obtained.

[0064] In one provided embodiment, road point cloud information is obtained; voxel features are extracted from the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information; obstacle detection is performed on the density-based voxel features and reflectivity-based multidimensional feature vectors through a voxel neural network model to obtain road obstacle detection results, and by setting the voxel value as a multidimensional feature vector of density and reflectivity, the information loss of the point cloud is reduced, thereby improving the detection accuracy of the model.

[0065] Step S30 , obtaining the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to adjacent voxels based on the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels.

[0066] In the specific implementation, the features of adjacent voxels are combined into a large feature to reduce the amount of data processed by the model. For example, feature_111 represents the voxel feature with coordinates (1, 1, 1), and the adjacent voxels are voxels with coordinates (1, 2, 1), (2, 1, 1), and (2, 2, 1), which are represented by feature_121, feature_211, and feature_221 respectively.

[0067] Step S40 , synthesizing the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain target voxel features corresponding to each of the voxels.

[0068] In a specific implementation, the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels are integrated to obtain an integrated multidimensional feature vector; the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels are updated according to the integrated multidimensional feature vectors of each of the voxels to obtain the target voxel features corresponding to each of the voxels, wherein the target voxel feature is a new voxel feature, represented by new_feature_111, and new_feature_111 is the new voxel feature at the coordinates (1, 1, 1) after the combination. Voxel features, the synthesis of density-based voxel features and reflectivity-based multidimensional feature vectors and the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels are exemplified, new_feature_111 = [feature_111, feature_121, feature_211, feature_221], 111, 121, 211, 221 represent voxel coordinates with coordinates (1, 1, 1), (1, 2, 1), (2, 1, 1), (2, 2, 1), respectively, and new_feature_111 is the new voxel feature with coordinates (1, 1, 1) after the combination. This processing can reduce the number of features from (H*W*Z) to (H / 2*W / 2*Z), and the number of channels increases from 3 to 3*4=12.

[0069] Step S50: Obstacle detection is performed on the target voxel features through a voxel neural network model to obtain a road obstacle detection result.

[0070] In the provided embodiment, in order to further reduce the information loss of the point cloud and not increase the amount of calculation exponentially, a "3d spatial2channel" scheme is provided to synthesize multiple adjacent voxels into a "large voxel". Although the number of channels of the "large voxel" will increase, it only increases the amount of calculation of the first layer of the network, and will not significantly increase the overall calculation amount of the model, thereby improving the calculation efficiency of the model.

[0071] Reference Figure 5 , Figure 5 The flowchart of the second embodiment of the obstacle detection method of the present invention is as follows. The second embodiment of the present invention is proposed based on the first embodiment. In the second embodiment, step S20 includes:

[0072] Step S201, determining the target point cloud quantity and the actual point cloud quantity according to the road point cloud information.

[0073] In a specific implementation, the three-dimensional space where the road point cloud information is located is divided to obtain voxels; the point cloud information contained in each voxel is determined according to the center point and range of each voxel; the target point cloud quantity and the actual point cloud quantity are determined according to the point cloud information contained in the voxel. For example, the point cloud is divided by Density_Reflectivity VFE, and the three-dimensional space is divided into (H*W*Z) voxels. The point cloud quantity in each voxel is calculated according to the center point and voxel size of each voxel.

[0074] Furthermore, determining the target point cloud quantity and the actual point cloud quantity according to the point cloud information contained in the voxel includes:

[0075] A current voxel is randomly selected from each of the voxels; point cloud information contained in the current voxel is determined; the number of point clouds in the point cloud information contained in the current voxel is counted; the number of point clouds in the current voxel is used as the target number of point clouds; and the number of point clouds in other voxels that is smaller than the target number of point clouds is used as the actual number of point clouds.

[0076] In the specific implementation, the target point cloud number is the maximum point cloud number, which is represented by max_point_num. After obtaining the point cloud of each voxel, max_point_num points are randomly selected as the representative point cloud of the voxel. If there are less than max_point_num points, the actual number is used, and the actual number is represented by n.

[0077] Step S202 , performing voxel feature extraction according to the target point cloud quantity and the actual point cloud quantity, and obtaining density-based voxel features corresponding to each voxel in the road point cloud information.

[0078] The density-based voxel features corresponding to each voxel in the road point cloud information are obtained by dividing the actual point cloud number by the target point cloud number, that is, Density feature extraction, density = n / max_point_num, where the point cloud number n satisfies 0<=n<=max_point_num.

[0079] In the provided embodiment, after the Density_Reflectivity VFE performs point cloud division, it performs downsampling. After obtaining the point cloud of each voxel, it randomly selects max_point_num points as the representative point cloud of the voxel and performs density feature extraction on the voxels with less than max_point_num point clouds to obtain the density information of the point cloud, so as to improve the detection accuracy when performing obstacle detection.

[0080] In one embodiment, the step S20 includes:

[0081] Extracting the reflectivity of the point cloud corresponding to each voxel in the road point cloud information; generating a multidimensional feature vector according to the reflectivity; and obtaining a multidimensional feature vector based on the reflectivity according to the multidimensional feature vector.

[0082] It should be noted that the multidimensional feature vector based on reflectivity can be a multidimensional feature vector or a two-dimensional feature vector, without limitation. Taking a two-dimensional feature vector as an example, the voxel feature based on reflectivity is represented by reflectivity_vector, then reflectivity_vector = [reflectivity_1, reflectivity_2], where reflectivity_1 = reflectivity / 255, reflectivity_2 = 1-reflectivity_1. By combining density and reflectivity, the final feature = [density, reflectivity_1, reflectivity_2].

[0083] In general, when obstacle detection is performed through reflectivity, the reflectivity value is directly transmitted to the neural network. However, the reflectivity is processed in this embodiment by converting the reflectivity into a multi-dimensional vector, which is more in line with the physical meaning of the reflectivity itself, making it easier for the model to extract useful information from the point cloud, thereby improving the accuracy of model detection.

[0084] Further, generating a multidimensional feature vector according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity includes:

[0085] Determine the number of point clouds corresponding to each of the voxels; when the number of point clouds corresponding to each of the voxels reaches the target number, average the reflectivity of the point clouds corresponding to each of the voxels to obtain an average reflectivity; and generate a multidimensional feature vector based on the reflectivity according to the average reflectivity.

[0086] It is understandable that if there are multiple points in a voxel, the average values ​​of reflectivity_1 and reflectivity_2 are calculated respectively. Reflectivity refers to the reflectivity of the point cloud, which is generally highly correlated with the material of the object. It can be considered that those greater than 100 are high-reflectivity materials, and those less than 100 are ordinary materials. Therefore, reflectivity_vector can reflect the category of the object contained in the voxel to a certain extent. The disadvantage of directly passing reflectivity into the neural network is that it turns a classification problem into a regression problem, which increases the difficulty of model learning.

[0087] In this embodiment, objects with low reflectivity and high reflectivity can be distinguished by reflectivity, that is, by defining reflectivity_2=1-reflectivity_1, the relationship between the reflectivity two-dimensional vectors is obtained. For example, if the reflectivity is 0, the two-dimensional vector is (0,1), and if the reflectivity is 255, the two-dimensional vector is (1,0). It can be seen from the reflectivity feature that there is a one-to-one correspondence from low reflectivity to high reflectivity, that is, the transition from (0,1) to (1,0). Since objects with low reflectivity and high reflectivity can be distinguished by reflectivity, the extracted features are more conducive to the learning of the neural network, thereby improving the accuracy of model detection.

[0088] In the specific implementation, a point cloud feature extractor for voxels with richer information is provided, which combines density and reflectivity, and reduces the input size through 3D spatial2channel operations to improve the real-time performance of the model. The reflectivity two-dimensional vector proposed in this embodiment is a soft_label, that is, the value range is 0 to 1, and a multi-dimensional vector can also be generated, and it is a hard_label, that is, the value of each dimension is 0 or 1, and only one dimension is 1. At the same time, a soft_label larger than two dimensions can also be generated, and the method is similar to the two-dimensional soft_label, which will not be repeated here.

[0089] The present invention further provides an obstacle detection device.

[0090] Reference Figure 6 , Figure 6 FIG. 1 is a schematic diagram of functional modules of the first embodiment of the obstacle detection device of the present invention.

[0091] In a first embodiment of the obstacle detection device of the present invention, the obstacle detection device comprises:

[0092] The acquisition module 10 is used to acquire road point cloud information.

[0093] The extraction module 20 is used to extract voxel features from the road point cloud information to obtain density-based voxel features and reflectivity-based multi-dimensional feature vectors corresponding to each voxel in the road point cloud information.

[0094] The detection module 30 is used to obtain the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to adjacent voxels according to the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels.

[0095] The detection module 30 is further used to synthesize the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain the target voxel features corresponding to each of the voxels.

[0096] The detection module 30 is also used to perform obstacle detection on the target voxel features through a voxel neural network model to obtain a road obstacle detection result.

[0097] In the provided embodiment, in order to further reduce the information loss of the point cloud and not increase the amount of calculation exponentially, a "3d spatial2channel" scheme is provided to synthesize multiple adjacent voxels into a "large voxel". Although the number of channels of the "large voxel" will increase, it only increases the amount of calculation of the first layer of the network, and will not significantly increase the overall calculation amount of the model, thereby improving the calculation efficiency of the model.

[0098] Optionally, the extraction module 20 is further used to determine the number of target point clouds and the number of actual point clouds according to the road point cloud information;

[0099] Voxel features are extracted according to the target point cloud quantity and the actual point cloud quantity to obtain density-based voxel features corresponding to each voxel in the road point cloud information.

[0100] Optionally, the extraction module 20 is further used to divide the three-dimensional space where the road point cloud information is located to obtain various voxels;

[0101] Determining the point cloud information contained in each voxel according to the center point and range of each voxel;

[0102] The target point cloud quantity and the actual point cloud quantity are determined according to the point cloud information contained in the voxel.

[0103] Optionally, the extraction module 20 is further configured to randomly select a current voxel from each of the voxels;

[0104] Determining point cloud information contained in the current voxel;

[0105] Counting the number of point clouds in the point cloud information contained in the current voxel;

[0106] The number of point clouds of the current voxel is used as the target point cloud number;

[0107] The number of point clouds in other voxels that is smaller than the target number of point clouds is taken as the actual number of point clouds.

[0108] Optionally, the extraction module 20 is further used to extract the reflectivity of the point cloud corresponding to each voxel in the road point cloud information;

[0109] A multidimensional feature vector is generated according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity.

[0110] Optionally, the extraction module 20 is further used to determine the number of point clouds corresponding to each of the voxels;

[0111] When the number of point clouds corresponding to each of the voxels reaches a target number, averaging the reflectivity of the point clouds corresponding to each of the voxels to obtain an average reflectivity;

[0112] A reflectivity-based multi-dimensional feature vector is generated according to the average reflectivity.

[0113] In addition, to achieve the above-mentioned purpose, the present invention also proposes an obstacle detection device, which includes: a memory, a processor, and an obstacle detection program stored in the memory and executable on the processor, wherein the obstacle detection program is configured to implement the obstacle detection method described above.

[0114] Since the obstacle detection device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0115] In addition, an embodiment of the present invention further provides a storage medium, on which an obstacle detection program is stored. When the obstacle detection program is executed by a processor, the obstacle detection method as described above is implemented.

[0116] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0117] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0118] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling an intelligent terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0120] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An obstacle detection method, characterized in that: The obstacle detection method comprises: Get road point cloud information; Extracting voxel features from the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information; According to the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels, the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels are obtained; Synthesizing the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain target voxel features corresponding to each of the voxels; Using the target voxel features to perform obstacle detection through a voxel neural network model to obtain a road obstacle detection result; The step of synthesizing the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain the target voxel features corresponding to each of the voxels includes: Integrating the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain an integrated multidimensional feature vector; The density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each voxel are updated according to the multidimensional feature vectors after integration of each voxel to obtain the target voxel features corresponding to each voxel, wherein the length and width of the target voxel features are 1 / 2 of the length and width of each voxel, and the number of synthesized channels is updated to the original number of channels*(voxel+number of adjacent voxels).

2. The obstacle detection method according to claim 1, characterized in that: Extracting voxel features from the road point cloud information to obtain density-based voxel features corresponding to each voxel in the road point cloud information includes: Determine the target point cloud quantity and the actual point cloud quantity according to the road point cloud information; Voxel features are extracted according to the target point cloud quantity and the actual point cloud quantity to obtain density-based voxel features corresponding to each voxel in the road point cloud information.

3. The obstacle detection method according to claim 2, characterized in that: The determining the target point cloud quantity and the actual point cloud quantity according to the road point cloud information includes: Dividing the three-dimensional space where the road point cloud information is located to obtain individual voxels; Determining the point cloud information contained in each voxel according to the center point and range of each voxel; The target point cloud quantity and the actual point cloud quantity are determined according to the point cloud information contained in the voxel.

4. The obstacle detection method according to claim 3, characterized in that: The step of determining the target point cloud quantity and the actual point cloud quantity according to the point cloud information contained in the voxel includes: Randomly select a current voxel from each of the voxels, where the current voxel is a voxel corresponding to the maximum number of point clouds obtained by randomly selecting multiple times from each of the voxels; Determining point cloud information contained in the current voxel; Counting the number of point clouds in the point cloud information contained in the current voxel; The number of point clouds of the current voxel is used as the target point cloud number; The number of point clouds in other voxels that is smaller than the target number of point clouds is taken as the actual number of point clouds.

5. The obstacle detection method according to any one of claims 1 to 4, characterized in that: The step of extracting voxel features from the road point cloud information to obtain a reflectivity-based multidimensional feature vector corresponding to each voxel in the road point cloud information includes: Extracting the reflectivity of the point cloud corresponding to each voxel in the road point cloud information; A multidimensional feature vector is generated according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity.

6. The obstacle detection method according to claim 5, characterized in that: The step of generating a multidimensional feature vector according to the reflectivity to obtain a multidimensional feature vector based on the reflectivity includes: Determine the number of point clouds corresponding to each of the voxels; When the number of point clouds corresponding to each of the voxels reaches a target number, averaging the reflectivity of the point clouds corresponding to each of the voxels to obtain an average reflectivity; A reflectivity-based multi-dimensional feature vector is generated according to the average reflectivity.

7. An obstacle detection device, characterized in that: The obstacle detection device comprises: Acquisition module, used to obtain road point cloud information; An extraction module, used for performing voxel feature extraction on the road point cloud information to obtain density-based voxel features and reflectivity-based multidimensional feature vectors corresponding to each voxel in the road point cloud information; A detection module, configured to obtain the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to adjacent voxels according to the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels; The detection module is further used to synthesize the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain target voxel features corresponding to each of the voxels; The detection module is further used to perform obstacle detection on the target voxel features through a voxel neural network model to obtain a road obstacle detection result; The detection module is further used to integrate the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels and the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to the adjacent voxels to obtain an integrated multidimensional feature vector; update the density-based voxel features and the reflectivity-based multidimensional feature vectors corresponding to each of the voxels according to the integrated multidimensional feature vectors of each of the voxels to obtain a target voxel feature corresponding to each of the voxels, wherein the number of lengths and widths of the target voxel features is 1 / 2 of the number of lengths and widths of each of the voxels, and the number of synthesized channels is updated to the original number of channels*(number of voxels+number of adjacent voxels).

8. An obstacle detection device, characterized in that: The obstacle detection device comprises: a memory, a processor, and an obstacle detection program stored in the memory and executable on the processor, wherein the obstacle detection program is configured to implement the obstacle detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores an obstacle detection program, and when the obstacle detection program is executed by the processor, the obstacle detection method according to any one of claims 1 to 6 is implemented.

Images