Obstacle detection method, apparatus and electronic device

Abstract

The application discloses an obstacle detection method and device and electronic equipment. It is related to the field of artificial intelligence, and the method comprises the following steps: acquiring a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located; randomly selecting a plurality of pixels in the two-dimensional image; determining target pixels from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixels are pixels corresponding to obstacles in the environment in which the target vehicle is located; performing back projection processing on the target pixels to obtain target three-dimensional point clouds, wherein the target three-dimensional point clouds are used to supplement the number of point clouds of the three-dimensional point clouds; and detecting the three-dimensional point clouds and the target three-dimensional point clouds to obtain obstacle information. The application solves the technical problem of low driving safety caused by the fact that laser radar is prone to missing obstacles in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular, to an obstacle detection method and device and electronic equipment. BACKGROUND

[0002] The core capability of an intelligent driving vehicle is that it can dynamically acquire and identify surrounding environment information in real time, which includes but is not limited to the state of the vehicle, traffic flow information, road conditions, traffic signs, etc. In order to achieve the above intelligent level, intelligent driving vehicles often install external perception components such as millimeter wave radars and laser radars, and corresponding dynamic perception algorithms.

[0003] However, laser radar sensors lag behind traditional color cameras in terms of resolution and cost. For intelligent driving algorithms, large objects close to the laser radar are easy to see, but far or small objects are easy to miss due to resolution and other reasons, thereby causing safety accidents.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] The embodiments of the present application provide an obstacle detection method, device and electronic equipment to at least solve the technical problem of low driving safety caused by the easy missed detection of obstacles by laser radars in the prior art.

[0006] According to an aspect of the embodiments of the present application, an obstacle detection method is provided, comprising: acquiring a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located; randomly selecting a plurality of pixels in the two-dimensional image; determining a target pixel from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel is a pixel corresponding to an obstacle in the environment in which the target vehicle is located; performing back projection processing on the target pixel to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement the number of point clouds of the three-dimensional point cloud; and detecting the three-dimensional point cloud and the target three-dimensional point cloud to obtain obstacle information.

[0007] Further, the obstacle detection method further comprises: determining at least one target object in the two-dimensional image before randomly selecting a plurality of pixels in the two-dimensional image; and performing division processing on the two-dimensional image according to the object contour of the at least one target object to obtain a target two-dimensional image composed of a plurality of image regions.

[0008] Further, the obstacle detection method further comprises: projecting the three-dimensional point cloud into the target two-dimensional image to obtain a projection pixel set composed of a plurality of projection pixels; and randomly selecting a plurality of pixels in the target two-dimensional image in a case where the number of projection pixels is less than or equal to a first preset number.

[0009] Further, the obstacle detection method further comprises: performing random sampling in the at least one image region to obtain a sample pixel set corresponding to the at least one image region, wherein the sample pixel set comprises a plurality of sample pixels.

[0010] Further, the obstacle detection method further comprises: determining a target projection pixel set corresponding to the at least one sample pixel set, wherein the target projection pixel set is in the same image region as the sample pixel set; calculating a sum of distances between the sample pixels in the sample pixel set and each projection pixel in the target projection pixel set to obtain distance information; and determining a target pixel from the sample pixels based on the distance information corresponding to each sample pixel.

[0011] Further, the obstacle detection method further comprises: after determining the target pixel from the plurality of pixels according to the three-dimensional point cloud, performing the following steps: step 1, counting a pixel number of the determined target pixel; step 2, randomly selecting a plurality of current pixels in the two-dimensional image in a case where the pixel number of the target pixel is less than a second preset number; step 3, determining a target current pixel from the plurality of current pixels according to the three-dimensional point cloud; step 4, adding the pixel number of the target current pixel and the pixel number of the target pixel to obtain a target pixel number; step 5, updating the pixel number of the target pixel based on the target pixel number; and step 6, repeating the steps 1 to 5 until the pixel number of the target pixel is greater than or equal to the second preset number, and performing back projection processing on the target pixel and the target current pixel to obtain the target three-dimensional point cloud.

[0012] According to another aspect of the embodiments of the present application, an obstacle detection device is also provided, comprising: an acquisition module configured to acquire a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located; a selection module configured to randomly select a plurality of pixels in the two-dimensional image; a determination module configured to determine a target pixel from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel is a pixel corresponding to an obstacle in the environment in which the target vehicle is located; a processing module configured to perform back projection processing on the target pixel to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement a point cloud number of the three-dimensional point cloud; and a detection module configured to detect the three-dimensional point cloud and the target three-dimensional point cloud to obtain obstacle information.

[0013] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is configured to execute the above-mentioned obstacle detection method when running.

[0014] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes one or more processors; and a memory storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement a program for running, wherein the program is configured to perform the above-mentioned obstacle detection method when running.

[0015] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes computer programs / instructions, which when executed by a processor, implement the above-mentioned obstacle detection method.

[0016] In the embodiments of the present application, the original three-dimensional point cloud is enriched based on the pixels in the two-dimensional image, and the obstacle information is determined based on the enriched point cloud information. In the embodiments of the present application, the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located are obtained, a plurality of pixels are randomly selected in the two-dimensional image, a target pixel is determined from the plurality of pixels according to the three-dimensional point cloud, the target pixel is subjected to back projection processing to obtain a target three-dimensional point cloud, and finally the three-dimensional point cloud and the target three-dimensional point cloud are detected to obtain the obstacle information. The target pixel is a pixel corresponding to an obstacle in the environment where the target vehicle is located, and the target three-dimensional point cloud is used to supplement the number of point clouds of the three-dimensional point cloud.

[0017] In the above process, the plurality of pixels are randomly selected in the two-dimensional image, which avoids the problems of strong subjectivity of manual selection of pixels or low scene applicability of intelligent selection of pixels, and ensures that there is no image area that is easily ignored in the two-dimensional image in the process of selecting pixels. Further, since the three-dimensional point cloud can accurately represent the distribution of the obstacle, the target pixel is determined according to the three-dimensional point cloud, which realizes accurate determination of the target pixel corresponding to the obstacle. Further, since the distant objects or small objects in the two-dimensional image are more clearly visible relative to the three-dimensional point cloud, the target three-dimensional point cloud used to enrich the three-dimensional point cloud is generated based on the target pixel in the two-dimensional image, which makes up for the defect that the detection device such as a laser radar is prone to missing point clouds in the process of detecting distant objects or small objects to generate a three-dimensional point cloud, thereby improving the detection capability of the method for the obstacle in the case of missing the original three-dimensional point cloud, and further improving the safety of vehicle driving.

[0018] As can be seen, the scheme provided in the present application achieves the purpose of enriching the original three-dimensional point cloud based on the pixels in the two-dimensional image, and determining the obstacle information based on the enriched point cloud information, thereby realizing the technical effect of improving the safety of vehicle driving, and further solving the technical problem of low driving safety caused by the laser radar missing the obstacle in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0020] Figure 1 is a schematic diagram of an optional obstacle detection method according to an embodiment of the application;

[0021] Figure 2 is a schematic diagram of an optional obstacle detection device according to an embodiment of the application. DETAILED DESCRIPTION

[0022] In order to make the persons skilled in the art better understand the application scheme, the technical scheme in the embodiments of the application will be described clearly and completely below in conjunction with the drawings of the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by the persons skilled in the art without creative work should belong to the protection scope of the application.

[0023] It should be noted that the terms "first", "second", and the like in the specification and claims of the application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] Embodiment 1

[0025] According to the embodiments of the application, an embodiment of an obstacle detection method is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0026] Figure 1 is a schematic diagram of an optional obstacle detection method according to an embodiment of the application, as Figure 1 shown, the method comprises the following steps:

[0027] In step S102, the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located are acquired.

[0028] In step S102, the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located can be acquired by an electronic device, a server, an application system or the like. In this embodiment, the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located are acquired by an optional detection system. The information collection field for generating the three-dimensional point cloud and the two-dimensional image can be at least one of the front environment, the side environment and the rear environment of the target vehicle, or can be the environment around the target vehicle, i.e., the information collection field is 360°.

[0029] Optionally, a detection device for generating the three-dimensional point cloud and an image collection device for generating the two-dimensional image can be arranged on the target vehicle. The detection system includes a collection module which can send a data collection instruction to the detection device and the image collection device to collect the relevant three-dimensional point cloud and two-dimensional image, or receive the relevant three-dimensional point cloud and two-dimensional image sent by the detection device and the image collection device in real time, so as to realize the acquisition of the three-dimensional point cloud and the two-dimensional image.

[0030] In step S104, a plurality of pixels are randomly selected in the two-dimensional image.

[0031] In step S104, the detection system can directly randomly select a plurality of pixels in the two-dimensional image according to a preset random selection algorithm, or can randomly select a plurality of pixels in the two-dimensional image after a certain preprocessing, and then randomly select a plurality of pixels in the preprocessed two-dimensional image according to the preset random selection algorithm. The preprocessing can be region division, image enhancement, object recognition, feature extraction or the like.

[0032] It should be noted that by randomly selecting a plurality of pixels in the two-dimensional image, the problem of strong subjectivity of manual selection of pixels or low scene applicability of intelligent selection of pixels is avoided, so that it is ensured that there is no image area that is easily ignored in the two-dimensional image in the process of pixel selection, thereby facilitating the improvement of the richness effect of the three-dimensional point cloud.

[0033] In step S106, a target pixel is determined from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel is a pixel corresponding to an obstacle in the environment where the target vehicle is located.

[0034] In step S106, the detection system can determine the pixels relatively close to the three-dimensional point cloud as target pixels from the plurality of pixels. Specifically, the detection system can first project the three-dimensional point cloud into a two-dimensional image to obtain projected pixels corresponding to the three-dimensional point cloud, then determine the sum of distances between each pixel and each projected pixel, and determine the target pixels based on the sum of distances. Alternatively, the detection system can also determine the target pixels as the pixels existing within a target distance range of each projected pixel after obtaining the projected pixels corresponding to the three-dimensional point cloud. The target pixels determined above are the pixels corresponding to the obstacles in the environment of the target vehicle.

[0035] It should be noted that since the three-dimensional point cloud can accurately represent the distribution of obstacles, the target pixels are determined based on the three-dimensional point cloud, which realizes accurate determination of the target pixels, thereby facilitating effective enrichment of the three-dimensional point cloud.

[0036] In step S108, the target pixels are subjected to back projection processing to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement the point cloud quantity of the three-dimensional point cloud.

[0037] Alternatively, the detection system further comprises a simulated 3D point cloud generation module, which can project the target pixels into a three-dimensional space based on the back projection matrix or other back projection methods to obtain a target three-dimensional point cloud corresponding to the target pixels.

[0038] It should be noted that the distant objects or small objects in the two-dimensional image are more clearly visible relative to the three-dimensional point cloud, and therefore, the target three-dimensional point cloud used to enrich the three-dimensional point cloud is generated based on the target pixels in the two-dimensional image, which compensates for the defect that the detection device such as the laser radar is prone to missing point clouds in the process of detecting distant objects or small objects to generate a three-dimensional point cloud, thereby improving the detection ability of the method for obstacles in the case of missing original three-dimensional point clouds, and realizing a dual-mode based obstacle detection method.

[0039] In step S110, the three-dimensional point cloud and the target three-dimensional point cloud are detected to obtain obstacle information.

[0040] In step S110, the detection system further comprises a 3D target detection module. Optionally, the 3D target detection module can take the union of the three-dimensional point cloud and the target three-dimensional point cloud to obtain a total three-dimensional point cloud, and process the total three-dimensional point cloud by using a 3D point cloud obstacle detection algorithm, so that the obstacle information in the environment where the target vehicle is located can be obtained. Optionally, the 3D target detection module can also process the total three-dimensional point cloud based on a machine learning model or other existing mature 3D detection algorithm to obtain the aforementioned obstacle information. The obstacle information at least includes the distribution of the obstacle relative to the target vehicle, the shape of the obstacle, the distance between the obstacle and the target vehicle, and the like. It should be noted that in the present embodiment, the obstacle is used to represent an object that may cause an obstacle to the driving of the target vehicle, and the obstacle includes but is not limited to pedestrians, animals, other vehicles, signs, vegetation, and the like.

[0041] It should be noted that by detecting the supplemented point cloud information, accurate and relatively complete obstacle information can be obtained, thereby improving the safety of vehicle driving.

[0042] Based on the scheme defined in steps S102 to S110, it can be known that in the embodiment of the present application, the original three-dimensional point cloud is enriched based on the pixels in the two-dimensional image, and the obstacle information is determined based on the enriched point cloud information. By obtaining the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located, a plurality of pixels are randomly selected in the two-dimensional image, and then the target pixels are determined from the plurality of pixels according to the three-dimensional point cloud, the target pixels are subjected to back projection processing to obtain the target three-dimensional point cloud, and finally the three-dimensional point cloud and the target three-dimensional point cloud are detected to obtain the obstacle information. The target pixel is a pixel corresponding to an obstacle in the environment where the target vehicle is located, and the target three-dimensional point cloud is used to supplement the number of point clouds of the three-dimensional point cloud.

[0043] It is easy to note that in the above process, by randomly selecting a plurality of pixels in the two-dimensional image, the problem of strong subjectivity of manual selection of pixels or low scene applicability of intelligent selection of pixels is avoided, and it is ensured that there is no image area in the two-dimensional image that is easily ignored in the process of pixel selection. Further, since the three-dimensional point cloud can accurately represent the distribution of the obstacle, the target pixel corresponding to the obstacle is accurately determined by determining the target pixel according to the three-dimensional point cloud. Further, since the distant objects or small objects in the two-dimensional image are more clearly visible relative to the three-dimensional point cloud, the target three-dimensional point cloud used to enrich the three-dimensional point cloud is generated based on the target pixel in the two-dimensional image, which makes up for the defect that the detection device such as laser radar has in detecting distant objects or small objects to generate three-dimensional point cloud, thereby improving the detection ability of the present method for obstacles in the case of missing original three-dimensional point cloud, and further improving the safety of vehicle driving.

[0044] Therefore, the scheme provided in the application achieves the purpose of enriching the original three-dimensional point cloud based on the pixels in the two-dimensional image, and determining the obstacle information based on the enriched point cloud information, thereby realizing the technical effect of improving the safety of vehicle driving, and further solving the technical problem of low driving safety caused by the easy missed detection of obstacles by the laser radar in the prior art.

[0045] In an optional embodiment, before obtaining the three-dimensional point cloud and the two-dimensional image corresponding to the environment where the target vehicle is located, the acquisition module in the detection device can obtain a detection result of the detection device detecting the environment where the target vehicle is located, obtain the three-dimensional point cloud, and obtain an acquisition result of the image acquisition device acquiring the image of the environment where the target vehicle is located, and obtain the two-dimensional image.

[0046] Optionally, the detection device can be a laser radar or other device capable of generating a three-dimensional point cloud, and the image acquisition device can be a camera, a mobile phone or other device capable of generating a two-dimensional image. In this embodiment, the camera and the laser radar can be arranged on the target vehicle and simultaneously acquire scene data of the environment where the target vehicle is located to generate the corresponding two-dimensional image and three-dimensional point cloud. By utilizing the imaging advantages of the camera such as high resolution, low cost, rich color texture, etc., the two-dimensional image is generated to enhance and enrich the three-dimensional point cloud, which can effectively improve the detection capability in the case of missing original three-dimensional point cloud.

[0047] In an optional embodiment, before projecting the three-dimensional point cloud into the target two-dimensional image to obtain a projection pixel set composed of multiple projection pixels, the detection system can determine the relative pose of the detection device and the image acquisition device, and then determine the projection matrix and the back projection matrix based on the relative pose, wherein the back projection matrix is used for back projection processing of the target pixel.

[0048] Optionally, the detection system further comprises a calibration module. An inertial measurement unit IMU is arranged on the target vehicle, and the calibration module can use a calibration algorithm to take the inertial measurement unit IMU as an intermediate pose, and determine the extrinsic parameters of the detection device to the inertial measurement unit IMU and the extrinsic parameters of the image acquisition device to the inertial measurement unit IMU, so as to determine the relative pose of the detection device and the image acquisition device, and then determine the projection matrix for projecting the three-dimensional point cloud to the two-dimensional image and the back projection matrix for back projection processing of the target pixel based on the relative pose. The back projection matrix is the inverse matrix of the projection matrix. It should be noted that the calibration module can also take other units or modules as the intermediate pose to determine the relative pose of the detection device and the image acquisition device.

[0049] It should be noted that by determining the projection matrix and the inverse projection matrix, accurate determination of the correspondence between the two-dimensional image and the three-dimensional point cloud is achieved, avoiding the problem of being unable to accurately represent the obstacle information caused by projecting the three-dimensional point cloud or inversely projecting the target pixel without calibration.

[0050] In an optional embodiment, before randomly selecting the plurality of pixels in the two-dimensional image, the detection system can determine at least one target object in the two-dimensional image, and divide the two-dimensional image based on the object contour of the at least one target object to obtain the target two-dimensional image composed of a plurality of image regions.

[0051] Optionally, the detection system further comprises a 2D instance segmentation module. The 2D instance segmentation module can determine at least one target object in the two-dimensional image using an existing mature 2D image instance segmentation algorithm, where the target object can be a pedestrian, an animal, another vehicle, a sign, a plant, etc. Then, the 2D instance segmentation module can determine the object contour of the at least one target object, and divide the two-dimensional image based on the object contour of the target object to obtain the target two-dimensional image composed of a plurality of image regions. Each image region represents the object contour information of a target object.

[0052] It should be noted that by segmenting the two-dimensional image, the selection range of randomly selecting the plurality of pixels in the two-dimensional image is reasonably reduced, so that the target pixel can be more accurately and efficiently determined.

[0053] In an optional embodiment, during the process of randomly selecting the plurality of pixels in the two-dimensional image, the detection system can project the three-dimensional point cloud into the target two-dimensional image to obtain a projection pixel set composed of a plurality of projection pixels, and randomly select the plurality of pixels in the target two-dimensional image when the number of projection pixels is less than or equal to a first preset number.

[0054] The detection system can project the three-dimensional point cloud into the two-dimensional image or the target two-dimensional image using the projection matrix to obtain a projection pixel set composed of a plurality of projection pixels. Then, the detection system can count the number of projection pixels in the projection pixel set. When it is determined that the number of projection pixels is less than or equal to a first preset number, the plurality of pixels in the target two-dimensional image are randomly selected. Otherwise, if it is determined that the number of projection pixels is greater than the first preset number, the three-dimensional point cloud can be directly detected to obtain the obstacle information. The first preset number can be a fixed value, or can be determined based on the number of pixels of the two-dimensional image or the target two-dimensional image, and the number of target objects in the target two-dimensional image.

[0055] In another alternative embodiment, in the process of comparing the pixel number of the projected pixels with the first preset number, the detection system can directly determine the pixel number of the projected pixels projected into each image region of the target two-dimensional image, and determine whether the pixel number of the projected pixels corresponding to each image region is greater than a third preset number. In the case where it is determined that the pixel number of the projected pixels corresponding to a certain image region is less than or equal to the third preset number, only a plurality of pixels in the image region are randomly selected, otherwise, if it is determined that the pixel number of the projected pixels corresponding to all image regions is greater than the third preset number, the three-dimensional point cloud can be directly detected to obtain the obstacle information. The third preset number can be a fixed value, or can be determined based on the pixel number in the target two-dimensional image or the number of target objects.

[0056] In another alternative embodiment, in the process of comparing the pixel number of the projected pixels with the first preset number, the detection system can determine the pixel number of the projected pixels projected into each image region of the target two-dimensional image, and determine a fourth preset number corresponding to each image region based on the pixel number in each image region. Then, the detection system can determine whether the pixel number of the projected pixels corresponding to each image region is greater than the fourth preset number corresponding to the image region. In the case where it is determined that the pixel number of the projected pixels corresponding to a certain image region is less than or equal to the fourth preset number corresponding to the image region, only a plurality of pixels in the image region are randomly selected, otherwise, if it is determined that the pixel number of the projected pixels corresponding to all image regions is greater than the fourth preset number corresponding to the image region, the three-dimensional point cloud can be directly detected to obtain the obstacle information.

[0057] It should be noted that by judging the pixel number of the projected pixels of the three-dimensional point cloud to determine whether to supplement the three-dimensional point cloud, unnecessary resource occupation can be avoided, thereby improving the detection efficiency of the application for the obstacle.

[0058] In an alternative embodiment, in the process of randomly selecting a plurality of pixels in the two-dimensional image, the detection system can randomly sample in at least one image region to obtain a sample pixel set corresponding to the at least one image region, wherein the sample pixel set includes a plurality of sample pixels.

[0059] Specifically, in the case where it is determined that the pixel number of the projected pixels is less than or equal to the first preset number, the detection system can use an existing random sampling algorithm to randomly sample in at least one image region to obtain a sample pixel set corresponding to the at least one image region, and preferably, randomly sample in each image region.

[0060] Further, in the case where the pixel quantity of the projected pixels is determined to be less than or equal to the first preset quantity, the detection system can first determine the proportion of the pixel quantity of the projected pixels in the total pixel quantity in each image region, and only perform random sampling in the image regions corresponding to the projected pixels with a relatively low proportion to obtain the sampling pixel set corresponding to the image regions.

[0061] In another optional embodiment, in the case where it is determined that the pixel quantity of the projected pixels corresponding to a certain image region is less than or equal to the third preset quantity, the detection system can only perform random sampling in the image region using the existing random sampling algorithm to obtain the sampling pixel set corresponding to the image region.

[0062] In another optional embodiment, in the case where it is determined that the pixel quantity of the projected pixels corresponding to a certain image region is less than or equal to the fourth preset quantity corresponding to the image region, the detection system can only perform random sampling in the image region using the existing random sampling algorithm to obtain the sampling pixel set corresponding to the image region.

[0063] It should be noted that by performing random sampling in at least one image region, it is ensured that the image pixels corresponding to the target object in the two-dimensional image can be effectively selected, thereby improving the accuracy of random selection.

[0064] In an optional embodiment, in the process of determining the target pixel from the plurality of pixels according to the three-dimensional point cloud, the detection system can determine a target projected pixel set corresponding to at least one sampling pixel set, then calculate the sum of the distances between the sampling pixels in the sampling pixel set and each projected pixel in the target projected pixel set to obtain distance information, and determine the target pixel from the sampling pixels based on the distance information corresponding to each sampling pixel. The target projected pixel set and the sampling pixel set are in the same image region.

[0065] Optionally, taking one of the image regions in the two-dimensional image as an example, in the image region, the detection system can determine a target projected pixel set of the three-dimensional point cloud projected in the image region. A sampling pixel in the sampling pixel set corresponding to the image region is determined to calculate the Euclidean distance between the sampling pixel and each projected pixel in the target projected pixel set. Then, the Euclidean distances between the sampling pixel and each projected pixel in the target projected pixel set are added to obtain the distance information corresponding to the sampling pixel.

[0066] Similarly, for other sampling pixels in the image region, the detection system also uses the foregoing method to calculate to determine the distance information corresponding to the other sampling pixels.

[0067] Further, after determining the distance information corresponding to each sampling pixel in the image region, the detection system can determine the target pixel from the sampling pixels corresponding to the image region based on the distance information. In this case, the detection system can determine the sampling pixel corresponding to the distance information representing the shortest total distance as the target pixel, or sort the sampling pixels from near to far based on the total distances represented by the distance information, and determine the top n sampling pixels as the target pixels based on the number of sampling pixels in the image region.

[0068] Further, after determining the target pixels from all the image regions as described above, the determination of the target pixels in the current round of sampling pixels is achieved.

[0069] It should be noted that since the result of the three-dimensional point cloud has high accuracy, calculating the sum of distances between the sampling pixels in the sampling pixel set and each projection pixel in the target projection pixel set, and determining the target pixel from the sampling pixels based on the sum of distances, can effectively improve the accuracy of the target pixel, and further improve the accuracy of detecting obstacles.

[0070] In an alternative embodiment, after determining the target pixel from the plurality of pixels based on the three-dimensional point cloud, the detection system can perform the following steps:

[0071] Step 1, count the number of pixels of the determined target pixel;

[0072] Step 2, in the case where the number of pixels of the target pixel is less than a second preset number, randomly select a plurality of current pixels in the two-dimensional image;

[0073] Step 3, determine a target current pixel from the plurality of current pixels based on the three-dimensional point cloud;

[0074] Step 4, add the number of pixels of the target current pixel and the number of pixels of the target pixel to obtain a target pixel number;

[0075] Step 5, update the number of pixels of the target pixel based on the target pixel number;

[0076] Step 6, repeat steps 1 to 5 until the number of pixels of the target pixel is greater than or equal to the second preset number, and perform back projection processing on the target pixel and the target current pixel to obtain a target three-dimensional point cloud.

[0077] Optionally, the detection system can randomly select pixels and determine target pixels in the two-dimensional image for multiple rounds. Specifically, after the detection system determines the target pixels in the first round, the detection system can count the number of pixels of the target pixels determined in the current round and compare the number of pixels with the second preset number. In the case where the number of pixels of the target pixels is less than the second preset number, the detection system can start a new round of random selection of pixels in the two-dimensional image, i.e., select a current pixel. Specifically, the method of random selection of pixels is the same as the foregoing method, and thus is not described herein.

[0078] Further, after the detection system determines the current pixel selected in the new round, the detection system can determine the target current pixel in the round from the current pixel selected in the new round based on the foregoing determination method, and add the number of pixels of the target current pixel in the round to the number of pixels of the target pixels in all previous rounds to obtain the number of target pixels. Thus, the number of all target pixels determined at the current time can be determined based on the number of target pixels.

[0079] Further, when the number of all target pixels determined at the current time is still less than the second preset number in the new round, the detection system starts the next round of random selection of pixels and determination of target pixels. According to the foregoing process, after the target pixels are determined in each round, the detection system counts the number of target pixels determined and determines whether to start the next round according to the counting result, until the number of target pixels is greater than or equal to the second preset number. Then, the detection system performs back projection processing on all target pixels and target current pixels to obtain the target three-dimensional point cloud.

[0080] It should be noted that the number of target pixels is increased by performing multiple rounds of random selection of pixels and determination of target pixels, so that the original three-dimensional point cloud can be enriched better.

[0081] Therefore, the scheme provided in the present application achieves the purpose of enriching the original three-dimensional point cloud based on the pixels in the two-dimensional image and determining the obstacle information based on the enriched point cloud information, thereby achieving the technical effect of improving the safety of vehicle driving, and has the advantages of simplicity, high efficiency, low coupling, and wide application scenarios, thereby solving the technical problem of low driving safety caused by the easy missed detection of obstacles by the laser radar in the prior art.

[0082] Embodiment 2

[0083] According to the embodiment of the present application, an embodiment of an obstacle detection device is provided, wherein, Figure 2 is a schematic diagram of an optional obstacle detection device according to an embodiment of the present application, as Figure 2 shown, the device comprises:

[0084] The acquisition module 202 is configured to acquire a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located.

[0085] The selection module 204 is configured to randomly select a plurality of pixels in the two-dimensional image.

[0086] The determination module 206 is configured to determine a target pixel from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel corresponds to an obstacle in the environment in which the target vehicle is located.

[0087] The processing module 208 is configured to perform back-projection processing on the target pixel to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement the number of points in the three-dimensional point cloud.

[0088] The detection module 210 is configured to detect the three-dimensional point cloud and the target three-dimensional point cloud to obtain obstacle information.

[0089] It should be noted that the acquisition module, the selection module, the determination module, the processing module, and the detection module correspond to steps S202 to S210 in the above embodiment, and the five modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment 1.

[0090] Optionally, the obstacle detection apparatus further includes a first sub-determination module configured to determine at least one target object in the two-dimensional image; and a first sub-processing module configured to perform division processing on the two-dimensional image according to an object contour of the at least one target object to obtain a target two-dimensional image composed of a plurality of image regions.

[0091] Optionally, the selection module further includes a first projection module configured to project the three-dimensional point cloud into the target two-dimensional image to obtain a projection pixel set composed of a plurality of projection pixels; and a first sub-selection module configured to randomly select the plurality of pixels in the target two-dimensional image in a case where a pixel number of the projection pixels is less than or equal to a first preset number.

[0092] Optionally, the first sub-selection module further includes a random sampling module configured to perform random sampling in the at least one image region to obtain a sampling pixel set corresponding to the at least one image region, wherein the sampling pixel set includes a plurality of sampling pixels.

[0093] Optionally, the determination module further includes a second sub-determination module configured to determine a target projection pixel set corresponding to the at least one sampling pixel set, wherein the target projection pixel set and the sampling pixel set are in the same image region; a calculation module configured to calculate a sum of distances between the sampling pixels in the sampling pixel set and each projection pixel in the target projection pixel set to obtain distance information; and a third sub-determination module configured to determine the target pixel from the sampling pixels based on the distance information corresponding to each sampling pixel.

[0094] Optionally, the obstacle detection device further comprises an execution module configured to execute the following steps: step 1, counting the pixel number of the determined target pixels; step 2, randomly selecting a plurality of current pixels in the two-dimensional image in the case that the pixel number of the target pixels is less than a second preset number; step 3, determining target current pixels from the plurality of current pixels according to the three-dimensional point cloud; step 4, adding the pixel number of the target current pixels and the pixel number of the target pixels to obtain a target pixel number; step 5, updating the pixel number of the target pixels based on the target pixel number; and step 6, repeating the execution of steps 1 to 5 until the pixel number of the target pixels is greater than or equal to the second preset number, and then performing back projection processing on the target pixels and the target current pixels to obtain the target three-dimensional point cloud.

[0095] Embodiment 3

[0096] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is configured to execute the above-mentioned obstacle detection method when running.

[0097] Embodiment 4

[0098] According to another aspect of the embodiments of the present application, an electronic device is also provided, and the electronic device comprises one or more processors; a memory configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a program configured to execute the above-mentioned obstacle detection method when running.

[0099] Embodiment 5

[0100] According to another aspect of the embodiments of the present application, a computer program product is also provided, and the computer program product comprises computer programs / instructions, which, when executed by a processor, implement the above-mentioned obstacle detection method.

[0101] The above-mentioned serial numbers of the embodiments of the present application only serve for description, and do not represent the advantages or disadvantages of the embodiments.

[0102] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0103] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other manners. For example, the described unit embodiments can be divided into other ways, for example, the division of units can be a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between units can be indirect coupling or communication connection through some interfaces, and can be electrical or other forms.

[0104] The units described as separate components may or can not be physically separate, and the components shown as units may or can not be physical units, i.e., they can be located in one place, or can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0105] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0106] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0107] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. An obstacle detection method characterized by, The method comprises the following steps: acquiring a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located; randomly selecting a plurality of pixels in the two-dimensional image; determining a target pixel from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel corresponds to an obstacle in the environment in which the target vehicle is located; performing back projection processing on the target pixel to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement the point cloud quantity of the three-dimensional point cloud; detecting the three-dimensional point cloud and the target three-dimensional point cloud to obtain obstacle information; randomly selecting a plurality of pixels in the two-dimensional image comprises: projecting the three-dimensional point cloud into a target two-dimensional image to obtain a projection pixel set composed of a plurality of projection pixels, wherein the target two-dimensional image is obtained by performing division processing on the two-dimensional image; and in a case where the pixel quantity of the projection pixel is less than or equal to a first preset quantity, randomly selecting a plurality of pixels in the target two-dimensional image.

2. The method of claim 1, wherein, Before the step of randomly selecting a plurality of pixels in the two-dimensional image, the method comprises the following steps: determining at least one target object in the two-dimensional image; performing division processing on the two-dimensional image according to the object contour of the at least one target object to obtain a target two-dimensional image composed of a plurality of image regions.

3. The method of claim 1, wherein, In a case where the pixel quantity of the projection pixel is less than or equal to the first preset quantity, randomly selecting a plurality of pixels in the target two-dimensional image comprises: performing random sampling in at least one image region to obtain a sampling pixel set corresponding to the at least one image region, wherein the sampling pixel set comprises a plurality of sampling pixels.

4. The method of claim 3, wherein, Determining a target pixel from the plurality of pixels according to the three-dimensional point cloud comprises: determining a target projection pixel set corresponding to at least one sampling pixel set, wherein the target projection pixel set is in the same image region as the sampling pixel set; calculating the sum of distances between the sampling pixels in the sampling pixel set and each projection pixel in the target projection pixel set to obtain distance information; determining a target pixel from the sampling pixels based on the distance information corresponding to each sampling pixel.

5. The method of claim 1, wherein, After the step of determining a target pixel from the plurality of pixels according to the three-dimensional point cloud, the method further comprises the following steps: Step 1: counting the pixel quantity of the determined target pixel; Step 2: in a case where the pixel quantity of the target pixel is less than a second preset quantity, randomly selecting a plurality of current pixels in the two-dimensional image; Step 3: determining a target current pixel from the plurality of current pixels according to the three-dimensional point cloud; Step 4: adding the pixel quantity of the target current pixel and the pixel quantity of the target pixel to obtain a target pixel quantity; Step 5: updating the pixel quantity of the target pixel based on the target pixel quantity; Step 6: repeatedly performing steps 1 to 5 until the pixel quantity of the target pixel is greater than or equal to the second preset quantity, and performing back projection processing on the target pixel and the target current pixel to obtain a target three-dimensional point cloud.

6. An obstacle detection device characterized by comprising: The method comprises the following steps: The acquisition module is configured to acquire a three-dimensional point cloud and a two-dimensional image corresponding to an environment in which a target vehicle is located; The selection module is configured to randomly select a plurality of pixels in the two-dimensional image; The determination module is configured to determine a target pixel from the plurality of pixels according to the three-dimensional point cloud, wherein the target pixel corresponds to an obstacle in the environment in which the target vehicle is located; The processing module is configured to perform back-projection processing on the target pixel to obtain a target three-dimensional point cloud, wherein the target three-dimensional point cloud is used to supplement the number of points in the three-dimensional point cloud; The detection module is configured to detect the three-dimensional point cloud and the target three-dimensional point cloud to obtain obstacle information. The device is further configured to project the three-dimensional point cloud into a target two-dimensional image to obtain a set of projection pixels composed of a plurality of projection pixels, wherein the target two-dimensional image is obtained by performing division processing on the two-dimensional image; and randomly select a plurality of pixels in the target two-dimensional image when the number of pixels of the projection pixels is less than or equal to a first preset number.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is configured to execute the obstacle detection method in any one of claims 1 to 5 when running.

8. An electronic device, comprising: The electronic device includes one or more processors; The memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement a program for running, wherein the program is configured to execute the obstacle detection method in any one of claims 1 to 5 when running.

9. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the obstacle detection method in any one of claims 1 to 5.

Images