Obstacle identification method and device and medium

Abstract

The invention discloses an obstacle recognition method and device and a storage medium. The method comprises the steps that a target point cloud and a target image collected by an unmanned vehicle in a driving area at the current moment are acquired; extracting an actual obstacle point cloud from the target point cloud; converting the actual obstacle point cloud into a first obstacle image; extracting first obstacle feature information from the first obstacle image; extracting a second obstacle image from the target image; extracting second obstacle information feature information from the obstacle image; and comparing common feature information of the first obstacle feature information and the second obstacle feature information, and extracting an obstacle corresponding to the common feature information. The method can improve the extraction efficiency of the feature information in the obstacle image, improves the obstacle detection precision and detection efficiency, and solves the technical problems of huge calculation amount, excessive resource consumption, and obstacle missing detection and / or false detection caused by overlarge point cloud data amount in the obstacle detection process of a laser radar.

Description

Technical Field

[0001] The present invention relates to the technical field of obstacle recognition, and specifically to an obstacle recognition method, device and medium. Background Art

[0002] Obstacle recognition is one of the important research topics in the field of intelligent assisted driving and robotics applications. Generally, various types of sensors can be used for obstacle detection. Existing obstacle recognition has the following problems:

[0003] 1. The same obstacles identified by different types of sensors need to be associated. In the process of associating obstacles, different obstacles may be associated, resulting in incorrect associated obstacles, or the same obstacles may be missed, resulting in multiple identical obstacles.

[0004] 2. The point cloud data is sparse, which causes the point cloud data of objects that are far away and small in size to not accurately reflect the shape of the objects, resulting in low accuracy in obstacle recognition. Summary of the invention

[0005] In order to overcome the following problems in the existing obstacle recognition in the above-mentioned background technology: the same obstacles detected by different types of sensors need to be associated. In the process of associating obstacles, different obstacles may be associated, resulting in erroneous associated obstacles, and the same obstacles may be missed, resulting in multiple identical obstacles. The point cloud data is sparse, resulting in the point cloud data of distant objects and small objects not accurately reflecting the shape of the objects, resulting in low accuracy of obstacle recognition. The purpose of the present invention is to provide an obstacle recognition method.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides an obstacle recognition method, comprising the following steps:

[0008] Obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment;

[0009] Extracting an actual obstacle point cloud from the target point cloud;

[0010] Converting the actual obstacle point cloud into a first obstacle image;

[0011] Extracting first obstacle feature information from the first obstacle image;

[0012] Extracting a second obstacle image from the target image;

[0013] Extracting second obstacle information feature information from the obstacle image;

[0014] The common feature information of the first obstacle feature information and the second obstacle feature information is compared to extract the obstacle corresponding to the common feature information.

[0015] Compared with the prior art, the present invention has the following beneficial effects: by extracting first obstacle feature information from a first obstacle image converted from an actual obstacle point cloud, and extracting second obstacle feature information from a second obstacle image, the missed detection rate of obstacles can be reduced, and obstacles of different sizes and types can be comprehensively identified; by comparing common feature information of the first obstacle feature information and the second obstacle feature information, obstacles corresponding to the common feature information are extracted, and the efficiency of extracting feature information from the obstacle image can be improved, and the obstacle detection accuracy and detection efficiency can be improved, thereby solving the technical problems of huge calculation amount, excessive resource consumption, missed detection and / or false detection of obstacles caused by excessive amount of point cloud data in the process of obstacle detection by laser radar.

[0016] In some possible implementations, extracting the obstacle point cloud from the target point cloud specifically includes the following steps:

[0017] Using a standard division space range as a division basis for the target point cloud, the target point cloud is clipped to obtain a clipped point cloud, wherein the standard division space range is data obtained after setting the standard division space range in three directions of X, Y and Z in the three-dimensional map data;

[0018] Performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud;

[0019] Performing clustering and segmentation on the denoised point cloud to obtain a point cloud entity;

[0020] Obstacle feature matching is performed on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

[0021] In some possible implementations, extracting the first obstacle feature information from the first obstacle image specifically includes the following:

[0022] Matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result;

[0023] First obstacle feature information is generated according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

[0024] In some possible implementations, extracting the second obstacle image from the target image specifically includes the following steps:

[0025] Dividing the target image into a plurality of detection images;

[0026] Calculating the obstacle confidence of each of the detected images;

[0027] A second obstacle image in the target image is extracted according to the obstacle confidence.

[0028] A second aspect of the present invention provides an obstacle recognition device, comprising:

[0029] Data acquisition module: used to obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment;

[0030] Obstacle point cloud extraction module: used to extract the actual obstacle point cloud from the target point cloud;

[0031] A first obstacle image conversion module: used for converting the actual obstacle point cloud into a first obstacle image;

[0032] A first obstacle feature information extraction module: configured to extract first obstacle feature information from the first obstacle image;

[0033] A second obstacle image extraction module: used to extract a second obstacle image from the target image;

[0034] A second obstacle feature information extraction module: used to extract second obstacle information feature information from the obstacle image;

[0035] The obstacle common feature extraction module is used to compare the common feature information of the first obstacle feature information and the second obstacle feature information, and extract the obstacle corresponding to the common feature information.

[0036] In some possible implementations, extracting the obstacle point cloud from the target point cloud specifically includes the following steps:

[0037] Using a standard division space range as a division basis for the target point cloud, the target point cloud is clipped to obtain a clipped point cloud, wherein the standard division space range is data obtained after setting the standard division space range in three directions of X, Y and Z in the three-dimensional map data;

[0038] Performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud;

[0039] Performing clustering and segmentation on the denoised point cloud to obtain a point cloud entity;

[0040] Obstacle feature matching is performed on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

[0041] In some possible implementations, extracting the first obstacle feature information from the first obstacle image specifically includes the following:

[0042] Matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result;

[0043] First obstacle feature information is generated according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

[0044] In some possible implementations, extracting the obstacle image from the target image specifically includes the following steps:

[0045] Dividing the target image into a plurality of detection images;

[0046] Calculating the obstacle confidence of each of the detected images;

[0047] An obstacle image in the target image is extracted according to the obstacle confidence.

[0048] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above obstacle identification method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The following is a flowchart of the overall steps of the obstacle identification method according to an embodiment of the present invention;

[0050] Figure 2 This is a flowchart of the steps for extracting the actual obstacle point cloud according to an embodiment of the present invention;

[0051] Figure 3 This is a flow chart of the extraction steps of the first obstacle feature information according to an embodiment of the present invention;

[0052] Figure 4 Flowchart of the steps of extracting the second obstacle image according to an embodiment of the present invention; Figure 5 Schematic diagram of the structure of an obstacle recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0054] See attached Figure 1 As shown, the obstacle recognition method in this embodiment includes the following steps:

[0055] S1: Obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment.

[0056] S2: Extracting the actual obstacle point cloud from the target point cloud. Figure 2 Specifically, the steps include:

[0057] S21: using a standard partitioning space range as a partitioning basis for the target point cloud, cutting the target point cloud to obtain a cut point cloud, wherein the standard partitioning space range is data obtained after setting the standard partitioning space range in three directions of X, Y and Z in the three-dimensional map data;

[0058] S22: performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud;

[0059] S23: performing clustering segmentation on the denoised point cloud to obtain a point cloud entity;

[0060] S24: Perform obstacle feature matching on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

[0061] The point cloud belonging to the actual obstacle can be accurately extracted from the target point cloud, thereby improving the extraction accuracy and precision of the point cloud of the actual obstacle.

[0062] S3: Convert the actual obstacle point cloud into a first obstacle image.

[0063] S4: Extracting first obstacle feature information from the first obstacle image. Figure 3 Specifically, the steps include:

[0064] S41: matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result;

[0065] S42: Generate first obstacle feature information according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

[0066] By matching with the standard obstacle image template library, obstacles can be identified in multiple dimensions including obstacle distance information, obstacle speed information, obstacle orientation information, and obstacle size information, thereby reducing the missed detection rate.

[0067] S5: Extract the second obstacle image from the target image. Figure 4 Specifically, the steps include:

[0068] S51: dividing the target image into a plurality of detection images;

[0069] S52: Calculating the obstacle confidence of each of the detection images;

[0070] S53: Extracting a second obstacle image in the target image according to the obstacle confidence.

[0071] The obstacle confidence level can be used to accurately and comprehensively extract the second obstacle image from the target image, thereby improving the accuracy of obstacle recognition; and the efficiency of obstacle recognition can be accelerated by dividing the target image.

[0072] S6: Extracting second obstacle information feature information from the obstacle image.

[0073] S7: Compare common feature information of the first obstacle feature information and the second obstacle feature information, and extract obstacles corresponding to the common feature information.

[0074] By extracting the first obstacle feature information from the first obstacle image converted from the actual obstacle point cloud, and extracting the second obstacle feature information from the second obstacle image, the missed detection rate of obstacles can be reduced, and obstacles of different sizes and types can be comprehensively identified; by comparing the common feature information of the first obstacle feature information and the second obstacle feature information, the obstacles corresponding to the common feature information are extracted, the extraction efficiency of feature information in the obstacle image can be improved, and the obstacle detection accuracy and detection efficiency can be improved, thereby solving the technical problems of huge calculation amount, excessive resource consumption, missed detection and / or false detection of obstacles caused by excessive amount of point cloud data in the process of laser radar detecting obstacles.

[0075] This embodiment also provides an obstacle recognition device, which applies the above obstacle recognition method. Figure 5 As shown, the obstacle recognition device includes

[0076] Data acquisition module: used to obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment;

[0077] Obstacle point cloud extraction module: used to extract the actual obstacle point cloud from the target point cloud;

[0078] A first obstacle image conversion module: used for converting the actual obstacle point cloud into a first obstacle image;

[0079] A first obstacle feature information extraction module: configured to extract first obstacle feature information from the first obstacle image;

[0080] A second obstacle image extraction module: used to extract a second obstacle image from the target image;

[0081] A second obstacle feature information extraction module: used to extract second obstacle information feature information from the obstacle image;

[0082] The obstacle common feature extraction module is used to compare the common feature information of the first obstacle feature information and the second obstacle feature information, and extract the obstacle corresponding to the common feature information.

[0083] In some implementations, extracting the obstacle point cloud from the target point cloud specifically includes the following steps:

[0084] Using a standard division space range as a division basis for the target point cloud, the target point cloud is clipped to obtain a clipped point cloud, wherein the standard division space range is data obtained after setting the standard division space range in three directions of X, Y and Z in the three-dimensional map data;

[0085] Performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud;

[0086] Performing clustering and segmentation on the denoised point cloud to obtain a point cloud entity;

[0087] Obstacle feature matching is performed on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

[0088] In some implementations, extracting first obstacle feature information from the first obstacle image specifically includes the following:

[0089] Matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result;

[0090] First obstacle feature information is generated according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

[0091] In some implementations, extracting the obstacle image from the target image specifically includes the following steps:

[0092] Dividing the target image into a plurality of detection images;

[0093] Calculating the obstacle confidence of each of the detected images;

[0094] An obstacle image in the target image is extracted according to the obstacle confidence.

[0095] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above obstacle identification method are implemented.

[0096] The storage medium stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0097] The processor may also be referred to as a CPU (Central Processing Unit). The processor may be an integrated circuit chip with signal processing capabilities. The processor may also be:

[0098] DSP (Digital Signal Processor, digital signal processor, digital signal processor is a processor composed of large-scale or ultra-large-scale integrated circuit chips to complete certain signal processing tasks. It has gradually developed to meet the needs of high-speed real-time signal processing tasks. With the development of integrated circuit technology and digital signal processing algorithms, the implementation methods of digital signal processors are constantly changing, and the processing functions are constantly improving and expanding.)

[0099] ASIC (Application Specific Integrated Circuit) is an integrated circuit designed and manufactured to meet the needs of specific users and specific electronic systems.

[0100] FPGA (Field Programmable Gate Array) is a product further developed on the basis of programmable devices such as PAL (Programmable Array Logic) and GAL (generic array logic). It emerged as a semi-custom circuit in the field of application-specific integrated circuits (ASICs), which not only solves the shortcomings of custom circuits, but also overcomes the shortcomings of the limited number of gate circuits of the original programmable devices.

[0101] The general-purpose processor may be a microprocessor or any conventional processor. The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to allow people familiar with the technology to understand the content of the present invention and implement it, and cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An obstacle recognition method, characterized in that: The steps include: Obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment; Extracting an actual obstacle point cloud from the target point cloud; Converting the actual obstacle point cloud into a first obstacle image; Extracting first obstacle feature information from the first obstacle image; Extracting a second obstacle image from the target image; Extracting second obstacle information feature information from the obstacle image; The common feature information of the first obstacle feature information and the second obstacle feature information is compared to extract the obstacle corresponding to the common feature information.

2. The obstacle recognition method according to claim 1, characterized in that: Extracting obstacle point cloud from the target point cloud specifically includes the following steps: Using a standard division space range as a division basis for the target point cloud, the target point cloud is clipped to obtain a clipped point cloud, wherein the standard division space range is data obtained after setting the standard division space range in three directions of X, Y and Z in the three-dimensional map data; Performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud; Performing clustering and segmentation on the denoised point cloud to obtain a point cloud entity; Obstacle feature matching is performed on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

3. The obstacle recognition method according to claim 1, characterized in that: Extracting the first obstacle feature information from the first obstacle image specifically includes the following steps: Matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result; First obstacle feature information is generated according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

4. The obstacle recognition method according to claim 1, characterized in that: Extracting the second obstacle image from the target image specifically includes the following steps: Dividing the target image into a plurality of detection images; Calculating the obstacle confidence of each of the detected images; A second obstacle image in the target image is extracted according to the obstacle confidence.

5. The obstacle recognition device is characterized in that: include Data acquisition module: used to obtain the target point cloud and target image collected by the unmanned vehicle in the driving area at the current moment; Obstacle point cloud extraction module: used to extract the actual obstacle point cloud from the target point cloud; A first obstacle image conversion module: used for converting the actual obstacle point cloud into a first obstacle image; A first obstacle feature information extraction module: configured to extract first obstacle feature information from the first obstacle image; A second obstacle image extraction module: used to extract a second obstacle image from the target image; A second obstacle feature information extraction module: used to extract second obstacle information feature information from the obstacle image; The obstacle common feature extraction module is used to compare the common feature information of the first obstacle feature information and the second obstacle feature information, and extract the obstacle corresponding to the common feature information.

6. The obstacle recognition device according to claim 5, characterized in that: Extracting obstacle point cloud from the target point cloud specifically includes the following steps: Using a standard division space range as a division basis for the target point cloud, the target point cloud is clipped to obtain a clipped point cloud, wherein the standard division space range is data obtained after setting the standard division space range in three directions of X, Y and Z in the three-dimensional map data; Performing denoising sampling processing on the cropped point cloud to obtain a denoised point cloud; Performing clustering and segmentation on the denoised point cloud to obtain a point cloud entity; Obstacle feature matching is performed on the point cloud entity and the standard obstacle point cloud to extract the actual obstacle point cloud.

7. The obstacle recognition device according to claim 5, characterized in that: Extracting the first obstacle feature information from the first obstacle image specifically includes the following steps: Matching the first obstacle image with a standard obstacle image template library, and generating obstacle type data according to the matching result; First obstacle feature information is generated according to the obstacle type data, where the first obstacle feature information includes obstacle distance information, obstacle speed information, obstacle direction information, and obstacle size information.

8. The obstacle recognition device according to claim 5, characterized in that: Extracting the obstacle image from the target image specifically includes the following steps: Dividing the target image into a plurality of detection images; Calculating the obstacle confidence of each of the detected images; An obstacle image in the target image is extracted according to the obstacle confidence.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the obstacle identification method according to any one of claims 1 to 4 are implemented.

Images