A three-dimensional target detection method, device, equipment and storage medium

Abstract

The application provides a three-dimensional target detection method, device and equipment and a storage medium. An initial three-dimensional point cloud is obtained, the initial three-dimensional point cloud is divided into a plurality of grid regions, a two-dimensional feature image is generated based on a grid height feature value of each grid region, two-dimensional target detection is performed on the two-dimensional feature image, a two-dimensional target region detection frame position and a two-dimensional label are obtained, a three-dimensional voxel point cloud feature is determined according to the two-dimensional target region detection frame position and three-dimensional position information of a highest point cloud point of the initial three-dimensional point cloud in all grid regions, the three-dimensional voxel point cloud feature is input into a preset three-dimensional target detection model, three-dimensional coordinates of a three-dimensional target object are obtained, and detection of the three-dimensional target object is realized based on the obtained three-dimensional coordinates and the two-dimensional label. The image information is used to determine a target region belonging to the three-dimensional target object in the point cloud information in advance, accurate feature extraction is then performed, detection of the three-dimensional target object is realized, and the purpose of accurate recognition and positioning is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision perception and image processing, and in particular to a three-dimensional target detection method, device, equipment and storage medium. BACKGROUND

[0002] With the in-depth development of the industrial internet and the intelligent transformation and upgrading of Chinese industrial entities, intelligent application devices such as drones, driverless vehicles and industrial inspection robots have appeared in the field of industrial intelligent manufacturing in China. In the process of industrial intelligence, laser radar is a key sensor for replacing human visual perception for sensing operations. With the information obtained by the sensor, decision-making can be made through program instructions, and a full-process closed-loop unmanned operation can be achieved in the true sense.

[0003] However, in the prior art, visual sensors such as laser radar and optical cameras are mostly in the position of alarm monitoring and auxiliary manual judgment in industrial application scenarios. The reason is that the information obtained by the sensor is not fully used, and it is not possible to accurately obtain the attributes and positions of three-dimensional target objects like a person. For example, an optical camera can better recognize the target texture information (such as appearance contour, color, etc.) in the imaging area, but lacks stereoscopic positioning in depth space; the point cloud information generated by the laser radar sensor scanning can effectively show the three-dimensional information of the target area, but the disorder of the point cloud lacks the description of the attributes and contours of the three-dimensional target object. Therefore, based on any one recognition method, only part of the information of the three-dimensional target object can be obtained, and the three-dimensional spatial coordinates and object categories of the three-dimensional target object cannot be recognized at the same time. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a three-dimensional target detection method, device, equipment and storage medium to solve the technical problem that any one recognition method based on two-dimensional image recognition or three-dimensional radar scanning cannot recognize the three-dimensional spatial coordinates and object categories of the three-dimensional target object at the same time.

[0005] The application provides a three-dimensional target detection method, which comprises the following steps: obtaining an initial three-dimensional point cloud, wherein the initial three-dimensional point cloud comprises a target three-dimensional point cloud of a three-dimensional target object; dividing the initial three-dimensional point cloud into a plurality of grid regions, generating a two-dimensional feature image of the initial three-dimensional point cloud according to a grid height feature value of each grid region, wherein the grid height feature value is obtained based on three-dimensional position information of a highest point cloud point of the initial three-dimensional point cloud in the grid region; performing two-dimensional target detection on the two-dimensional feature image to obtain a two-dimensional target region detection frame position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object; determining a three-dimensional voxel point cloud feature of the three-dimensional target object according to the two-dimensional target region detection frame position and the three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all the grid regions; inputting the three-dimensional voxel point cloud feature into a preset three-dimensional target detection model to obtain a three-dimensional coordinate of the three-dimensional target object, and determining a spatial position of the three-dimensional target object and an object category of the three-dimensional target object based on the three-dimensional coordinate and the two-dimensional label, so as to complete the detection of the three-dimensional target object.

[0006] In an embodiment of the application, the initial three-dimensional point cloud is divided into a plurality of grid regions, which comprises: obtaining an initial cross-sectional image of the initial point cloud, and determining cross-sectional horizontal and vertical coordinate extreme values based on the cross-sectional image; generating a target cross-sectional image corresponding to the initial cross-sectional image based on the cross-sectional horizontal and vertical coordinate extreme values, and filling the missing part in the target cross-sectional image; dividing the target cross-sectional image into a plurality of standard grids, taking the standard grid as a grid cross section, and forming a grid region based on all point cloud points in the three-dimensional range of the plane where the grid cross section is located.

[0007] In an embodiment of the application, the two-dimensional feature image of the initial three-dimensional point cloud is generated based on the grid height feature value of each grid region, which comprises: obtaining a height value of the highest point cloud point of each grid region, and determining the height value of the highest point cloud point as the height feature value of the grid region, wherein the height feature values of at least two grid regions are different; assigning different layer identifiers to different height feature values, and generating a two-dimensional feature image based on the layer identifier.

[0008] In an embodiment of the present application, the two-dimensional feature image is subjected to two-dimensional target detection to obtain a two-dimensional target region detection frame position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object, comprising: obtaining a plurality of historical two-dimensional feature images, and labeling sample object regions and sample object labels in the historical two-dimensional feature images; training an initial convolution algorithm model based on the labeled historical two-dimensional feature images to obtain a two-dimensional image detection model; inputting the two-dimensional feature image into the two-dimensional image detection model to determine a feature region and identify a label of the two-dimensional feature image, thereby obtaining a two-dimensional coordinate frame position of the three-dimensional target object in the two-dimensional feature image and a two-dimensional label of the three-dimensional target object.

[0009] In an embodiment of the present application, the three-dimensional voxel point cloud feature of the three-dimensional target object is determined according to the two-dimensional target region detection frame position and three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all the grid regions, comprising: determining a region as a target feature region in the two-dimensional feature image based on the two-dimensional target region detection frame position, and obtaining the highest point cloud point of each grid region in the target feature region based on the mapping relationship between the two-dimensional feature image and the three-dimensional position information of the highest point cloud point of all the grid point clouds, wherein the target feature region comprises at least one grid region; generating a highest point cloud point three-dimensional coordinate set based on the obtained highest point cloud point of each grid region in the target feature region, and obtaining a lowest point cloud point in a target three-dimensional region based on the highest point cloud point three-dimensional coordinate set, wherein the target three-dimensional region is a three-dimensional region in which the three-dimensional target object is located; determining a highest one of the highest point cloud points of each grid region as a target highest point cloud point in the target three-dimensional region; constructing a three-dimensional voxel cuboid with the position of the lowest point cloud point as the lowest point, the position of the target highest point cloud point as the highest point, and the two-dimensional target region detection frame as the cross section, and determining the three-dimensional voxel cuboid as the three-dimensional voxel point cloud feature of the three-dimensional target object.

[0010] In an embodiment of the present application, the two-dimensional feature image of the initial three-dimensional point cloud is generated based on the grid height feature value of each grid region, further comprising: obtaining the height feature value of each grid region, and obtaining the maximum feature value and the minimum feature value among the height feature values of all the grid regions; dividing the grid regions into a plurality of height levels based on the maximum feature value and the minimum feature value; determining a height level in which the height feature value of a target grid region is located, and determining the gray scale of the target grid region based on the height level to obtain the gray scale of each grid region; generating a two-dimensional gray scale image based on the obtained gray scale of each grid region, and determining the gray scale two-dimensional image as the two-dimensional feature image of the initial three-dimensional point cloud.

[0011] In an embodiment of the present application, before the three-dimensional voxel point cloud feature is input into the preset three-dimensional target detection model, the method further comprises: obtaining a plurality of historical three-dimensional voxel point cloud features, and labeling the three-dimensional voxel point cloud features based on three-dimensional coordinates and three-dimensional labels of a three-dimensional target object, the historical three-dimensional voxel point features being three-dimensional voxel point features of the three-dimensional target object; training a three-dimensional convolutional neural network based on the labeled historical three-dimensional voxel point cloud features to obtain a three-dimensional target detection model.

[0012] In an embodiment of the present application, after the three-dimensional convolutional neural network is trained based on the labeled historical three-dimensional voxel point cloud features to obtain a three-dimensional target detection model, the method further comprises: inputting a three-dimensional voxel point cloud feature to be detected into the three-dimensional target detection model to obtain three-dimensional coordinates and a three-dimensional label of the three-dimensional target object; if the three-dimensional label is different from a two-dimensional label of the three-dimensional target object, labeling the three-dimensional voxel point cloud feature based on the two-dimensional label, and updating training the three-dimensional convolutional neural network based on the labeled three-dimensional voxel point cloud feature to obtain an updated three-dimensional target detection model.

[0013] The present application provides a three-dimensional target detection device, which comprises: an information acquisition module for acquiring an initial three-dimensional point cloud, the initial three-dimensional point cloud comprising a target three-dimensional point cloud of a three-dimensional target object; a point cloud gridding module for dividing the initial three-dimensional point cloud into a plurality of grid regions, generating a two-dimensional feature image of the initial three-dimensional point cloud based on a grid height feature value of each grid region, the grid height feature value being obtained based on three-dimensional position information of a highest point cloud point of the initial three-dimensional point cloud in the grid region; a two-dimensional image processing module for performing two-dimensional target detection on the two-dimensional feature image to obtain a two-dimensional target region bounding box position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object; a three-dimensional point cloud mapping module for determining a three-dimensional voxel point cloud feature of the three-dimensional target object according to the two-dimensional target region bounding box position and three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all the grid regions; and a three-dimensional target object detection module for determining a three-dimensional coordinate of the three-dimensional target object based on the three-dimensional voxel point cloud feature and a preset three-dimensional target detection model, and determining a spatial position of the three-dimensional target object and an object category of the three-dimensional target object based on the three-dimensional coordinate and the two-dimensional label, so as to complete detection of the three-dimensional target object.

[0014] The present application provides an electronic device, which comprises: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the three-dimensional target detection method as described above.

[0015] The application provides a computer readable storage medium, characterized in that a computer program is stored thereon, and the computer program, when executed by a processor of a computer, causes the computer to perform the three-dimensional target detection method.

[0016] The three-dimensional target detection method, device, equipment and storage medium provided by the application have the following beneficial effects: the initial point cloud information containing three-dimensional target object information is acquired, the initial point cloud is gridded to obtain a plurality of point cloud grids, the two-dimensional feature image is generated based on the height feature values of the point cloud grids, the two-dimensional coordinate frame and the two-dimensional label of the three-dimensional target object are obtained by extracting the features of the two-dimensional feature image, the three-dimensional grid of the target image is obtained based on the two-dimensional coordinate frame and the two-dimensional feature image, the voxelized cuboid of the three-dimensional target object is obtained based on the three-dimensional grid, the three-dimensional target detection model is constructed based on the two-dimensional label and the voxelized cuboid, the voxelized cuboid containing the three-dimensional target object information is input into the three-dimensional target detection model, and the three-dimensional coordinates and the object category of the three-dimensional target object are obtained; the point cloud image is gridded to obtain the two-dimensional feature image, the prior image knowledge is used to guide and quickly locate the three-dimensional target detection, the two-dimensional image recognition and the three-dimensional scanning are effectively combined, the image information is used to tell the positions of the point cloud that belong to the three-dimensional target object, the accurate feature extraction is performed, the detection and recognition of the three-dimensional target object in the three-dimensional space are realized, the advantages of the image and the point cloud features are fully exerted, and the purpose of accurate recognition and positioning is achieved.

[0017] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings incorporated by reference in the specification and forming a part thereof illustrate embodiments consistent with the present application and together with the specification serve to explain the principles of the application. It is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings. In the drawings:

[0019] Figure 1 is a schematic diagram of an implementation environment of the three-dimensional target detection method according to an exemplary embodiment of the present application;

[0020] Figure 2 is a flowchart of the three-dimensional target detection method according to an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the initial point cloud in the three-dimensional target detection method according to an exemplary embodiment of the present application;

[0022] Figure 4 is a grid diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0023] Figure 5 is a two-dimensional feature image diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0024] Figure 6 is a two-dimensional coordinate frame diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0025] Figure 7 is a voxelized cuboid diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0026] Figure 8 is a three-dimensional object diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0027] Figure 9 is a flowchart of a three-dimensional target detection method according to an exemplary embodiment of the present application;

[0028] Figure 10 is a block diagram of a three-dimensional target detection apparatus according to an exemplary embodiment of the present application;

[0029] Figure 11 shows a structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0030] Other advantages and effects of the present application can be easily understood by those skilled in the art from the above description of the embodiments of the present application. The present application can be implemented or applied in other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, and are not intended to limit the protection scope of the present application.

[0031] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only show the components related to the present application in the diagrams, but are not drawn according to the number, shape and size of the components in actual implementation. The shape, number and proportion of the components in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0032] In the following description, numerous specific details are discussed in order to provide a thorough explanation of embodiments of the application. It will be apparent, however, to one skilled in the art, that embodiments of the application can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring embodiments of the application.

[0033] First of all, it should be noted that image priori is generally obtained through a series of processes on natural images, and can obtain unique information that is different from other images.

[0034] YoloV5 (You Only Look Once version 5) is a target detection model used to identify and locate objects in videos or images. It is the latest version of the YOLO (You Only Look Once) algorithm, which is an improvement based on YOLOv4. It is an end-to-end deep learning model that can directly detect and locate targets from raw images. YoloV5 uses convolutional neural networks (CNN) to learn the features of objects in images, and uses multi-scale prediction and grid segmentation to detect and locate targets.

[0035] PointPillars (Point Pillar) is a fast target detection network based on point cloud, which is often used in the field of autonomous driving. It is a 3D fast target detection network that is grounded and widely used.

[0036] PCL (Point Cloud Library) is a large cross-platform open source C++ programming library that implements a large number of point cloud related general algorithms and efficient data structures, involving point cloud acquisition, filtering, segmentation, registration, retrieval, feature extraction, recognition, tracking, surface reconstruction, visualization, etc. It supports multiple operating system platforms and can run on Windows, Linux, Android, Mac OSX, and some embedded real-time systems.

[0037] Figure 1 is a real-time environment schematic diagram of the three-dimensional target detection method shown in an exemplary embodiment of the present application.

[0038] As Figure 1As shown, in the three-dimensional target detection process, a point cloud collection device 101 and a computer device 102 are involved, wherein the point cloud collection device 101 can be a laser radar scanning device, or any other device or equipment with a three-dimensional information collection function, and the present application does not make any limitation on this, and the computer device 102 can be at least one of a desktop graphic processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. A person skilled in the art can collect point cloud information of a target environment containing a three-dimensional target object through the point cloud collection device 101, and send the obtained point cloud information to the computer device 102 for processing. After the computer device 102 processes the collected point cloud information, the three-dimensional target object in the point cloud information is recognized, and the three-dimensional coordinates and object category of the three-dimensional target object are obtained.

[0039] Figure 2 An exemplary embodiment of the present application shows a three-dimensional target detection method flowchart.

[0040] As Figure 2 shown, in an exemplary embodiment, the three-dimensional target detection method at least includes steps S210 to S250, which are described in detail as follows:

[0041] Step S210, obtaining an initial three-dimensional point cloud, the initial three-dimensional point cloud including a target three-dimensional point cloud of a three-dimensional target object.

[0042] Figure 3 An exemplary embodiment of the present application shows an initial point cloud diagram in the three-dimensional target detection method, as Figure 3 shown, scanning a target environment containing a three-dimensional target object to obtain initial point cloud information, and there are multiple three-dimensional target objects in the initial point cloud.

[0043] In an embodiment of the present application, taking a laser radar device as an example of a three-dimensional point cloud information collection device. First, an environment space containing a three-dimensional target object is determined as a target space, and then the target space is scanned based on the laser radar device to obtain the laser point cloud of the target space, and further obtain the initial point cloud information, i.e. the above-mentioned initial three-dimensional point cloud.

[0044] It should be understood that based on the laser radar or any other three-dimensional point cloud collection device to collect the point cloud information of the three-dimensional target object, the two-dimensional image of the three-dimensional target object can not be obtained by relying on a camera or other optical sensor, but the required image contour information can be extracted from the above-mentioned initial point cloud information, and the initial point cloud of the target region can be collected by any three-dimensional point cloud collection device, including but not limited to a laser radar device, and the present application does not make any limitation on the three-dimensional point cloud collection device.

[0045] Step S220, divide the initial three-dimensional point cloud into a plurality of grid regions, generate a two-dimensional feature image of the initial three-dimensional point cloud based on the grid height feature value of each grid region, and the grid height feature value is obtained based on the three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in the grid region.

[0046] Figure 4 An exemplary embodiment of the present application shows a grid diagram of a three-dimensional target detection method, as shown in Figure 4 A straight line is drawn in the horizontal and vertical directions of the overhead view of the initial point cloud, and the initial point cloud is divided into a plurality of grids.

[0047] The initial three-dimensional point cloud is divided into a plurality of grid regions, including: obtaining an initial cross-sectional image of the initial point cloud, and determining cross-sectional horizontal and vertical coordinate extreme values based on the cross-sectional image; generating a target cross-sectional image corresponding to the initial cross-sectional image based on the cross-sectional horizontal and vertical coordinate extreme values, and filling the missing part in the target cross-sectional image; dividing the target cross-sectional image into a plurality of standard grids, and taking the standard grid as a grid cross section, forming a grid region based on the region where all point cloud points in the three-dimensional range of the grid cross section plane are located.

[0048] In an embodiment of the present application, taking a rectangular grid region as an example. First, obtain the initial point cloud information Point A from the laser radar device, then determine the horizontal coordinate x and the vertical coordinate y of the overhead view of the obtained initial point cloud information Point A , and then grid divide the initial point cloud based on the obtained horizontal and vertical coordinates and the preset image resolution. If the x-axis or y-axis division exceeds the dimension, the exceeding part can be filled with zero value. The formula of the grid division is as follows:

[0049]

[0050] Wherein, represents all values of the point cloud Point A in x and y dimensions, Min() represents the minimum value, Max() represents the maximum value, Pix represents the set image pixel resolution, and side represents the length of the side of the square grid.

[0051] Through such grid division, it can be ensured that each grid contains a plurality of point cloud points or filled all-zero points, and a grid set Grids n is formed, wherein n represents the total number of divided grids.

[0052] In another embodiment of the present application, after obtaining the horizontal and vertical coordinates of the initial point cloud information in the top view field, the missing part is filled in the horizontal and vertical coordinates to obtain a regular rectangular cross section, and then the obtained rectangular cross section is divided based on a preset standard grid, such as a regular hexagon with a side length of 5 cm, to obtain a plurality of grid cross sections with the same size as the standard grid and with non-overlapping coordinates. It should be understood that in the present application, the initial point cloud can be arbitrarily cut to obtain a plurality of grid cross sections, and the size or shape of the grid cross section is not limited. The grid cross section can be rectangular, circular, sector, or any polygon, and the grid cross section can be a regular image or an irregular image, and the plurality of grid cross sections can be the same or different. The present application does not limit the shape or size of the cross section.

[0053] It should be understood that based on the area of all point cloud points in the three-dimensional range of the plane where the grid cross section is located, a grid area is formed, including: determining any grid cross section as a target grid cross section, and based on the mapping relationship between the top view field initial cross section image and the initial three-dimensional point cloud, obtaining a target three-dimensional point cloud which has a mapping relationship with the target grid cross section, and determining the area where the target three-dimensional point cloud is located as a grid area, which is a cylinder with the target grid cross section as the cross section. The shape of the grid area is determined based on the grid cross section, so the present application also does not limit the shape or size of the grid area.

[0054] Figure 5 is a two-dimensional feature image schematic diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application. As shown in Figure 5 different height feature values are assigned different layer identifiers (gray values) to generate a two-dimensional feature image corresponding to each three-dimensional target object, and the colors of the grids in the two-dimensional feature image are different.

[0055] Generating a two-dimensional feature image of the initial three-dimensional point cloud based on the grid height feature value of each grid area includes: obtaining the height value of the highest point cloud point of each grid area, and determining the height value of the highest point cloud point as the height feature value of the grid area, and each point grid area includes at least two different height feature values; different height feature values are assigned different layer identifiers, and a two-dimensional feature image is generated based on the layer identifier.

[0056] In an embodiment of the present application, the generating of the two-dimensional feature image of the initial three-dimensional point cloud based on the grid height feature value of each grid region further comprises: obtaining the height feature value of each grid region, and obtaining the maximum feature value and the minimum feature value in the height feature value of each grid region; dividing the grid region into a plurality of height levels based on the maximum feature value and the minimum feature value; determining the height level of the height feature value of a target grid region, and determining the gray scale of the target grid region based on the height level to obtain the gray scale of each grid region; generating a two-dimensional gray scale image based on the obtained gray scale of each grid region, and determining the gray scale two-dimensional image as the two-dimensional feature image of the initial three-dimensional point cloud.

[0057] It should be understood that the maximum feature value is the maximum one of the plurality of height feature values of all grid regions, and the minimum feature value is the minimum one of the plurality of height feature values of all grid regions.

[0058] In an embodiment of the present application, the two-dimensional feature image can be constructed in any color or shape, and only the two-dimensional feature image represented by a gray scale is taken as an example. According to the above-mentioned grid division of the initial point cloud, each grid obtained includes at least one three-dimensional point cloud point, and the highest point cloud point in each grid is taken as the height reference value of the grid, and the spatial three-dimensional coordinates of the highest point are recorded, and the highest value and the lowest value of the entire point cloud point are calculated, and the height difference is divided into 0-255 height levels, and the gray scale value of the image 0-255 is corresponded, so that the height information is converted to the image gray scale information, and the gray scale two-dimensional image (i.e. the two-dimensional feature image) based on the three-dimensional overhead view is constructed, and the specific steps include:

[0059] S2201, based on the obtained grid set Grids n , the highest point cloud point in each grid is calculated:

[0060]

[0061] wherein, Max (Grids i ) z represents the point cloud point z value maximum point in the i-th grid, represents the xyz three-axis coordinate value of the point cloud point z value maximum point in the i-th grid.

[0062] Thus, the z-axis maximum point cloud point coordinate set corresponding to the n grids

[0063] S2202, based on the obtained point cloud Point A , the maximum value and the minimum value According to the height difference, the height and the gray value are correspondingly divided:

[0064]

[0065] Wherein, gray j area The height interval corresponding to the gray value j is The minimum value of the point cloud point on the z axis is represented by zmin, The maximum value of the point cloud point on the z axis is represented by zmax.

[0066] Through the relationship of gray mapping height, all height intervals are converted into a set of image gray information

[0067] In another embodiment of the present application, after the initial point cloud is divided into a plurality of grid regions, the height of each grid region is divided into a plurality of levels according to a preset number of height levels, and a specified position in each grid region is taken as a height feature value (such as the midpoint of the three-dimensional height of each grid region), and different icons are taken as layer identifiers of each height feature value, to generate a two-dimensional feature image including various different icons.

[0068] Therefore, in the process of converting the height feature of each grid region into a two-dimensional feature image, any point in each grid region can be selected as a height feature value, and any recognizable identifier can be used as a layer identifier, as long as the reference standards of the height feature values of each grid are the same, and each layer identifier can be converted into the spatial height of the grid region. The present application does not make any restrictions on the selection standards of the height feature values and the selection standards of the layer identifiers.S2203, based on the obtained grid set Grids n And the z-axis maximum point set recorded in each grid The point cloud Point A The grid block gray value divided according to the height under the field of view is represented by:

[0069]

[0070] Wherein, The gray value corresponding to the i-th grid block is represented by grayi, The z value of the point with the maximum z axis value in the i-th grid block is represented by zmaxi, The gray value corresponding to the z value is represented by grayz.

[0071] So that each grid block has a corresponding gray value, forming a gray two-dimensional image

[0072] Step S230, two-dimensional target detection is performed on the two-dimensional feature image to obtain a two-dimensional target region detection frame position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object.

[0073] The two-dimensional target detection on the two-dimensional feature image obtains a two-dimensional target region detection frame position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object, and includes: obtaining a plurality of historical two-dimensional feature images, and labeling sample object regions and sample object labels in the historical two-dimensional feature images; training an initial convolution algorithm model based on the labeled historical two-dimensional feature images to obtain a two-dimensional image detection model; inputting the two-dimensional feature image into the two-dimensional image detection model to determine a feature region and identify a label of the two-dimensional feature image, and obtaining a two-dimensional coordinate frame position of the three-dimensional target object in the two-dimensional feature image and a two-dimensional label of the three-dimensional target object.

[0074] Figure 6 An example embodiment of the present application shows a two-dimensional coordinate frame diagram of a three-dimensional target detection method. As shown in FIG. 6, the two-dimensional feature image is processed to obtain a two-dimensional coordinate frame corresponding to each three-dimensional target object and a two-dimensional label.

[0075] In an embodiment of the present application, taking the feature value of a gray two-dimensional image extracted based on a YoloV5 deep convolutional neural network as an example. The gray two-dimensional image is input into the YoloV5 deep convolutional neural network, and through feature extraction, a two-dimensional detection frame coordinate of a target region and a two-dimensional label are obtained, and the formula is as follows:

[0076]

[0077] Wherein, YoloV5() represents a target detection algorithm model, represents a two-dimensional coordinate of a detected target, and Class aim represents a class of a detected three-dimensional target object.

[0078] Step S240, determining a three-dimensional voxel point cloud feature of the three-dimensional target object according to the two-dimensional target region detection frame position and three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all grid regions.

[0079] Figure 7 An example embodiment of the present application shows a voxelized cuboid diagram of a three-dimensional target detection method. As shown in FIG. 7, based on a two-dimensional coordinate frame of any three-dimensional target object and a SIFT height corresponding to the two-dimensional coordinate of the two-dimensional coordinate frame, a voxelized cuboid is obtained. Figure 7

[0080] ​The three-dimensional voxel point cloud feature of the three-dimensional target object is determined according to the two-dimensional target region detection frame position and the three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all grid regions, including: determining a region as a target feature region in a two-dimensional feature image based on the two-dimensional target region detection frame position, and obtaining the highest point cloud point of each grid region in the target feature region based on the mapping relationship of the two-dimensional feature image and the three-dimensional position information of the highest point cloud point of all grid point clouds, and the target feature region includes at least one grid region; generating a highest point cloud point three-dimensional coordinate set based on the obtained highest point cloud point of each grid region in the target feature region, and obtaining a lowest point cloud point in a target three-dimensional region based on the highest point cloud point three-dimensional coordinate set and a point cloud library, the target three-dimensional region being a three-dimensional region where the three-dimensional target object is located; determining the highest one of the highest point cloud points of each grid region as a target highest point cloud point in the target three-dimensional region; taking the position of the lowest point cloud point as the lowest point, taking the position of the target highest point cloud point as the highest point, and taking the two-dimensional target region detection frame as the cross section, a three-dimensional voxel cuboid is constructed, and the three-dimensional voxel cuboid is determined as the three-dimensional voxel point cloud feature of the three-dimensional target object.

[0081] It should be understood that obtaining a lowest point cloud point in a target three-dimensional region based on the highest point cloud point three-dimensional coordinate set includes: determining each grid region as an integral region based on the obtained grid region, and obtaining a point cloud lowest point in the integral region based on the initial point cloud, which is the point cloud lowest point in the target three-dimensional region; or, determining the point cloud lowest point of each grid region based on the obtained grid region, and comparing the obtained multiple point cloud lowest points to obtain a lowest point cloud lowest point, and determining the lowest point cloud lowest point as the point cloud lowest point in the target three-dimensional region.

[0082] In an embodiment of the present application, the target three-dimensional grid is determined based on the above-mentioned gray two-dimensional image. Based on the obtained two-dimensional image coordinates, a three-dimensional grid is obtained, the coordinates correspond one-to-one to the coordinates of the two-dimensional image , so that the coordinate highest point of the target grid corresponding to the detection target can be obtained, and then a three-dimensional voxelization cuboid is constructed based on the obtained coordinate highest point of the target grid. The formula for determining the coordinate highest point of the target grid is as follows:

[0083]

[0084] wherein, the coordinates xy of the two-dimensional image correspond to a grid region in the Grids n set, and the three-dimensional coordinates of the highest point corresponding to the target grid are represented by XYZ.

[0085] Then, based on the obtained Given the three-dimensional coordinates of the highest point in the target region, use the PCL algorithm to find the point with the lowest z-axis value within this set of three-dimensional coordinates of the highest point. And obtain the height difference between the highest and lowest points. use xy axis coordinates and height difference Construct a cube encompassing the target detection region from a top-down field of view, and treat the resulting cube as a single entity to obtain a voxelized cube. If the point cloud... A If a system contains several detection targets, then it contains several voxelized cubes. Therefore, a set of cubes can be generated based on the voxelized cubes of each detection target, and this set of cubes can be named Voxel. Det .

[0086] Step S250: Based on the three-dimensional voxel point cloud features and the preset three-dimensional target detection model, determine the three-dimensional coordinates of the three-dimensional target object, and determine the spatial position and object category of the three-dimensional target object based on the three-dimensional coordinates and two-dimensional labels, so as to complete the detection of the three-dimensional target object.

[0087] Figure 8 This is a schematic diagram of a three-dimensional object illustrating a three-dimensional target detection method, as described in an exemplary embodiment of this application. Figure 8 As shown, after inputting the obtained voxelized rectangle into the 3D target detection model, the 3D coordinates of each 3D target object are obtained.

[0088] In one embodiment of the present invention, the three-dimensional coordinates of a three-dimensional target object are determined based on three-dimensional voxel point cloud features and a preset three-dimensional target detection model, and the three-dimensional position and object category of the three-dimensional target object are obtained based on the three-dimensional coordinates and two-dimensional labels. This includes: inputting the target three-dimensional voxel point cloud features of the three-dimensional target object into the three-dimensional target detection model to obtain the target three-dimensional coordinates of the three-dimensional target object; determining the target three-dimensional coordinates as the spatial position of the three-dimensional target object, and determining the two-dimensional labels as the object category of the three-dimensional target object.

[0089] It should be understood that before inputting the three-dimensional voxel point cloud features into the preset three-dimensional target detection model, the following steps are also included: acquiring multiple historical three-dimensional voxel point cloud features, and annotating the three-dimensional voxel point cloud features based on the three-dimensional coordinates and three-dimensional labels of the three-dimensional target object, wherein the historical three-dimensional voxel point features are the three-dimensional voxel point features of the three-dimensional target object; training the three-dimensional convolutional neural network based on the annotated historical three-dimensional voxel point cloud features to obtain the three-dimensional target detection model.

[0090] In an embodiment of the present application, taking the PointPillar three-dimensional point cloud target detection network as an example of a three-dimensional target detection model. The obtained three-dimensional voxel point cloud is input into the PointPillar three-dimensional point cloud target detection network, and through feature extraction and dimension transformation, the three-dimensional detection frame coordinates and object category information of the three-dimensional target object are obtained, thereby realizing the steps of three-dimensional target object recognition and positioning based on image priori. Specifically, the steps include:

[0091] Based on the obtained three-dimensional target object cube set named Voxel Det , the Class aim detected by the two-dimensional image is corresponded with the Voxel Det set, so that each cube voxel has a corresponding category label, and then the category label and Voxel Det features are input into the PointPillar three-dimensional point cloud target detection network:

[0092]

[0093] wherein PointPillar() represents the PointPillar three-dimensional point cloud target detection network, represents the three-dimensional space coordinates of the detected target, represents the category information of the detected target.

[0094] It should be understood that in the embodiment of the present application, the initial point cloud is processed by the image priori method, and then a two-dimensional feature image is generated according to the three-dimensional height feature value of the overhead view image, and based on the feature extraction and dimension conversion of the two-dimensional feature image, a three-dimensional grid of the three-dimensional target object is obtained, so that the initial point cloud with a large amount of data is processed into a three-dimensional voxel point cloud feature with a smaller amount of data, the amount of data in the three-dimensional model training and recognition process is reduced, the training time of the three-dimensional neural network (i.e. the above-mentioned dimension convolutional neural network) is greatly shortened, and the recognition accuracy of the trained three-dimensional detection model is improved by using the two-dimensional label as a constraint condition for the three-dimensional label.

[0095] It should be understood that based on the pico detection model, a two-dimensional label of the three-dimensional target object can be obtained, and based on the three-dimensional target detection model, a three-dimensional label of the three-dimensional target object can be obtained. Since the two-dimensional label and the three-dimensional label both indicate the object category of the same three-dimensional target object, the two-dimensional label and the three-dimensional label should be the same. If the two-dimensional label and the three-dimensional label are different, the two-dimensional label is used as the standard, and the three-dimensional detection model can be calibrated and trained based on the two-dimensional label to improve the accuracy of the three-dimensional target detection model.

[0096] In an embodiment of the present application, after the three-dimensional target detection model is obtained by training the three-dimensional convolutional neural network based on the labeled historical three-dimensional voxel point cloud features, the method further comprises: inputting the three-dimensional point cloud features to be detected into the three-dimensional target detection model to obtain the three-dimensional coordinates and the three-dimensional label of the three-dimensional target object; if the three-dimensional label and the two-dimensional label of the three-dimensional target object are different, labeling the three-dimensional voxel point cloud features based on the two-dimensional label, and updating and training the three-dimensional convolutional neural network based on the labeled three-dimensional voxel point cloud features to obtain an updated three-dimensional target detection model.

[0097] In an embodiment of the present application, the comparison of the obtained and the obtained two-dimensional label Class aim , if the predicted class information is different from Class aim , the prediction result is discarded, and the loss error in the updating training is updated with Class aim as the guide to obtain a prediction result with better accuracy.

[0098] Figure 9 is a work flow diagram of a three-dimensional target detection method according to an exemplary embodiment of the present application. As shown in Figure 9 , in an embodiment of the present application, to realize the recognition of a three-dimensional target object, first, the three-dimensional point cloud information of a region to be detected is obtained, then the point cloud is divided into region grids according to a set side length based on the point cloud overhead view field, the coordinate value of the highest point cloud point in each grid is taken as the gray feature value of the grid, the height information of the grid is converted into a gray value based on the gray feature value, a gray two-dimensional image (equivalent to a two-dimensional feature image) is generated, then the obtained gray two-dimensional image is input into a Yolov5 two-dimensional image target detection network to obtain the two-dimensional detection coordinates of the target region (i.e. the region where the three-dimensional target object is located), the type and the two-dimensional detection frame, and based on the coordinate mapping relationship between the two-dimensional detection frame and the gray two-dimensional image, a voxelized cube is obtained, finally the obtained cube is input into a PointPillar three-dimensional point cloud target detection network to obtain the three-dimensional label and the three-dimensional coordinates of the three-dimensional target object, realizing the recognition and positioning of the three-dimensional target object.

[0099] Figure 10 is a block diagram of a three-dimensional target detection device according to an exemplary embodiment of the present application. The device can be applied to Figure 1 the implementation environment shown in the figure, and is specifically configured in the computer device 102. The device can also be applied to other exemplary implementation environments, and is specifically configured in other devices, and the implementation environment to which the device is applied is not limited in the embodiment.

[0100] As shown in Figure 10As shown, the exemplary three-dimensional target detection apparatus includes: an information acquisition module 1010, a point cloud gridding module 1020, a two-dimensional image processing module 1030, a three-dimensional point cloud mapping module 1040, and a three-dimensional target object detection module 1050.

[0101] The information acquisition module 1010 is configured to acquire an initial three-dimensional point cloud, and the initial three-dimensional point cloud includes a target three-dimensional point cloud of a three-dimensional target object. The point cloud gridding module 1020 is configured to divide the initial three-dimensional point cloud into a plurality of grid regions, generate a two-dimensional feature image of the initial three-dimensional point cloud based on a grid height feature value of each grid region, and obtain the grid height feature value based on three-dimensional position information of a highest point cloud point of the initial three-dimensional point cloud in the grid region. The two-dimensional image processing module 1030 is configured to perform two-dimensional target detection on the two-dimensional feature image to obtain a two-dimensional target region detection frame position of the three-dimensional target object and a two-dimensional label of the three-dimensional target object. The three-dimensional point cloud mapping module 1040 is configured to determine a three-dimensional voxel point cloud feature of the three-dimensional target object according to the two-dimensional target region detection frame position and the three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all grid regions. The three-dimensional target object detection module 1050 is configured to determine a three-dimensional coordinate of the three-dimensional target object based on the three-dimensional voxel point cloud feature and a preset three-dimensional target detection model, and determine a spatial position of the three-dimensional target object and an object category of the three-dimensional target object based on the three-dimensional coordinate and the two-dimensional label, so as to complete detection of the three-dimensional target object.

[0102] It should be noted that the three-dimensional target detection apparatus provided in the above embodiments and the three-dimensional target detection method provided in the above embodiments belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, which will not be described here. In actual application, the functions of the above embodiments can be distributed by different functional modules to complete all or part of the functions described above, and this is not limited herein.

[0103] Embodiments of the present application also provide an electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the three-dimensional target detection method provided in each of the above embodiments.

[0104] Figure 11 The structure of a computer system of an electronic device suitable for implementing embodiments of the present application is shown. It should be noted that, Figure 11 The computer system 1100 of the electronic device shown is only an example, and should not limit the functions and use range of the embodiments of the present application.

[0105] like Figure 11 As shown, the computer system 1100 includes a Central Processing Unit (CPU) 1101, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1102 or programs loaded from storage portion 1108 into Random Access Memory (RAM) 1103, such as performing the methods described in the above embodiments. The RAM 1103 also stores various programs and data required for system operation. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An Input / Output (I / O) interface 1105 is also connected to the bus 1104.

[0106] The following components are connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0107] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by central processing unit (CPU) 1101, it performs various functions defined in the system of this application.

[0108] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer-readable computer program is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit the program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted in any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0109] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems, which perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0110] The units described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware, and the described units can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0111] Another aspect of the present application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor of a computer, the computer performs the three-dimensional target detection method as described above. The computer readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device.

[0112] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the three-dimensional target detection method provided in each of the above embodiments.

[0113] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A three-dimensional target detection method, characterized in that, The method includes: Acquire an initial 3D point cloud, wherein the initial 3D point cloud includes the target 3D point cloud of the 3D target object; The initial 3D point cloud is divided into multiple grid regions, and a 2D feature image of the initial 3D point cloud is generated based on the grid height feature value of each grid region. The grid height feature value is obtained based on the 3D position information of the highest point of the initial 3D point cloud in the grid region. Two-dimensional target detection is performed on the two-dimensional feature image to obtain the position of the two-dimensional target region detection box of the three-dimensional target object and the two-dimensional label of the three-dimensional target object; The three-dimensional voxel point cloud features of the three-dimensional target object are determined based on the position of the detection box of the two-dimensional target region and the three-dimensional position information of the highest point cloud point of the initial three-dimensional point cloud in all the grid regions. The three-dimensional voxel point cloud features are input into a preset three-dimensional target detection model to obtain the three-dimensional coordinates of the three-dimensional target object. Based on the three-dimensional coordinates and the two-dimensional label, the spatial position and object category of the three-dimensional target object are determined to complete the detection of the three-dimensional target object. The three-dimensional voxel point cloud features of the three-dimensional target object are determined based on the position of the detection box in the two-dimensional target region and the three-dimensional position information of the highest point cloud point in the initial three-dimensional point cloud in all the grid regions, including: Based on the position of the two-dimensional target region detection box, a region is determined in the two-dimensional feature image as the target feature region. Based on the mapping relationship between the two-dimensional feature image and the three-dimensional position information of the highest point cloud points of all grid point clouds, the highest point cloud points of each grid region in the target feature region are obtained. The target feature region includes at least one grid region. Based on the highest point cloud points of each grid region in the obtained target feature region, a set of three-dimensional coordinates of the highest point cloud points is generated, and the three-dimensional voxel point cloud features of the three-dimensional target object are determined according to the set of three-dimensional coordinates of the highest point cloud points.

2. The three-dimensional target detection method according to claim 1, characterized in that, The initial 3D point cloud is divided into multiple grid regions, including: Obtain an initial cross-sectional image of the initial 3D point cloud, and determine the extreme values ​​of the cross-sectional horizontal and vertical coordinates based on the initial cross-sectional image; Based on the extreme values ​​of the horizontal and vertical coordinates of the cross section, a target cross section image corresponding to the initial cross section image is generated, and the missing parts in the target cross section image are filled in; The target cross-sectional image is divided into multiple standard grids, and a grid region is formed based on the area of ​​all point cloud points within the three-dimensional range of the plane where the standard grid cross-section is located, using the standard grid as the grid cross-section.

3. The three-dimensional target detection method according to claim 1, characterized in that, Generating a two-dimensional feature image of the initial three-dimensional point cloud based on the grid height feature value of each grid region includes: Obtain the height value of the highest point cloud point in each grid region, and determine the height value of the highest point cloud point as the height feature value of the grid region. There are at least two grid regions with different height feature values. Different layer identifiers are assigned to different height feature values, and a two-dimensional feature image is generated based on the layer identifiers.

4. The three-dimensional target detection method according to claim 1, characterized in that, Performing two-dimensional target detection on the two-dimensional feature image to obtain the two-dimensional target region detection box position of the three-dimensional target object and the two-dimensional label of the three-dimensional target object includes: Multiple historical two-dimensional feature images are acquired, and the sample object regions and sample object labels in the historical two-dimensional feature images are labeled. The initial convolutional algorithm model is trained based on the labeled historical two-dimensional feature images to obtain a two-dimensional image detection model; The two-dimensional feature image is input into the two-dimensional image detection model to determine the feature region and identify the label in the two-dimensional feature image, thereby obtaining the two-dimensional coordinate frame position of the three-dimensional target object in the two-dimensional feature image and the two-dimensional label of the three-dimensional target object.

5. The three-dimensional target detection method according to claim 1, characterized in that, The three-dimensional voxel point cloud features of the three-dimensional target object are determined based on the set of three-dimensional coordinates of the highest point cloud points, including: Based on the set of three-dimensional coordinates of the highest point cloud point, a lowest point cloud point is obtained within the target three-dimensional region, where the target three-dimensional region is the three-dimensional region where the three-dimensional target object is located. The highest point among the highest point cloud points in each grid region is determined as a target highest point cloud point in the target 3D region; Using the position of the lowest point cloud point as the lowest point and the position of the highest point cloud point of the target as the highest point, and using the two-dimensional target region detection box as the cross section, a three-dimensional voxel rectangle is constructed, and the three-dimensional voxel rectangle is determined as the three-dimensional voxel point cloud feature of the three-dimensional target object.

6. The three-dimensional target detection method according to claim 1, characterized in that, Generating a two-dimensional feature image of the initial three-dimensional point cloud based on the grid height feature value of each grid region further includes: Obtain the height feature value of each grid region, and get the maximum and minimum feature values ​​among the height feature values ​​of all grid regions; Based on the maximum and minimum eigenvalues, the grid region is divided into multiple height levels; Determine the height level of the height feature value of a target grid region, and determine the gray level of the target grid region based on the height level to obtain the gray level of each grid region; Based on the grayscale of each obtained grid region, a two-dimensional grayscale image is generated, and the two-dimensional grayscale image is determined as the two-dimensional feature image of the initial three-dimensional point cloud.

7. The three-dimensional target detection method according to any one of claims 1-6, characterized in that, Before inputting the three-dimensional voxel point cloud features into the preset three-dimensional target detection model, the following steps are also included: Multiple historical 3D voxel point cloud features are acquired, and the 3D voxel point cloud features are labeled based on the 3D coordinates and 3D labels of the 3D target object. The historical 3D voxel point cloud features are the 3D voxel point features of the 3D target object. A 3D convolutional neural network is trained based on the labeled historical 3D voxel point cloud features to obtain a 3D object detection model.

8. The three-dimensional target detection method according to claim 7, characterized in that, After training a 3D convolutional neural network based on labeled historical 3D voxel point cloud features to obtain a 3D object detection model, the following steps are also included: The three-dimensional voxel point cloud features to be detected are input into the three-dimensional target detection model to obtain the three-dimensional coordinates and three-dimensional labels of the three-dimensional target object; If the 3D label and the 2D label of the 3D target object are different, the 3D voxel point cloud features are labeled based on the 2D label, and the 3D convolutional neural network is updated and trained based on the labeled 3D voxel point cloud features to obtain the updated 3D target detection model.

9. A three-dimensional target detection device, characterized in that, The device includes: The information acquisition module is used to acquire an initial three-dimensional point cloud, which includes the target three-dimensional point cloud of the three-dimensional target object; A point cloud meshing module is used to divide the initial three-dimensional point cloud into multiple mesh regions, and generate a two-dimensional feature image of the initial three-dimensional point cloud based on the mesh height feature value of each mesh region. The mesh height feature value is obtained based on the three-dimensional position information of the highest point of the initial three-dimensional point cloud in the mesh region. A two-dimensional image processing module is used to perform two-dimensional target detection on the two-dimensional feature image to obtain the position of the two-dimensional target region detection box of the three-dimensional target object and the two-dimensional label of the three-dimensional target object; The 3D point cloud mapping module is used to determine the 3D voxel point cloud features of the 3D target object based on the position of the detection box of the 2D target region and the 3D position information of the highest point of the initial 3D point cloud in all the grid regions. The three-dimensional target object detection module is used to input the three-dimensional voxel point cloud features into a preset three-dimensional target detection model to obtain the three-dimensional coordinates of the three-dimensional target object, and determine the spatial position and object category of the three-dimensional target object based on the three-dimensional coordinates and the two-dimensional label, so as to complete the detection of the three-dimensional target object; The three-dimensional point cloud mapping module is specifically used for: Based on the position of the two-dimensional target region detection box, a region is determined in the two-dimensional feature image as the target feature region. Based on the mapping relationship between the two-dimensional feature image and the three-dimensional position information of the highest point cloud points of all grid point clouds, the highest point cloud points of each grid region in the target feature region are obtained. The target feature region includes at least one grid region. Based on the highest point cloud points of each grid region in the obtained target feature region, a set of three-dimensional coordinates of the highest point cloud points is generated, and the three-dimensional voxel point cloud features of the three-dimensional target object are determined according to the set of three-dimensional coordinates of the highest point cloud points.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the three-dimensional target detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the three-dimensional target detection method according to any one of claims 1 to 8.

Images