Target detection method and device, vehicle control unit and readable storage medium

Abstract

The embodiment of the invention provides a target detection method and device, a vehicle control unit and a readable storage medium, and relates to the technical field of target detection, and the method comprises the steps: carrying out the feature extraction of a voxel space determined based on point cloud data, and obtaining a spatial three-dimensional feature; performing feature extraction on the two-dimensional image to obtain a two-dimensional to-be-fused feature; performing feature extraction on the spatial three-dimensional features and the two-dimensional to-be-fused features based on a three-dimensional backbone network to obtain target extraction features output by the three-dimensional backbone network; and performing target detection on the target extraction features based on the region-of-interest pooling network, and fusing a pooling result generated in the target detection process with the two-dimensional to-be-fused features to obtain a three-dimensional target detection result output by the region-of-interest pooling network. According to the invention, the accuracy of three-dimensional target detection can be improved through the complementation of the point cloud information and the two-dimensional image in the process of three-dimensional target detection according to the point cloud data.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method, device, vehicle controller and readable storage medium. Background Art

[0002] Relying on long-term technological accumulation and recent breakthroughs in the field of artificial intelligence, coupled with increasingly mature policies, driverless cars are coming to people. The fact that major Internet companies and automobile giants have invested heavily in the development of driverless technology shows that driverless cars have gradually become a general trend in the information industry and the automobile industry. Among the many key technologies of driverless cars, the perception technology of the traffic environment is undoubtedly the core and foundation.

[0003] In the problem of dynamic target perception in unmanned driving, because the dynamic targets in the driving environment are complex and diverse, and contain many human factors, this requires unmanned vehicles to have a strong perception of the actual driving environment. The relevant technologies usually include two-dimensional target detection technology and three-dimensional target detection technology. Among them, the two-dimensional target detection technology uses two-dimensional images taken by visual cameras to detect targets such as traffic signs, lane lines, and signal lights; the three-dimensional target detection technology detects targets based on point cloud images taken by laser radar.

[0004] However, the original information used by the above target detection technologies is relatively simple, resulting in poor accuracy in target detection. Summary of the invention

[0005] In view of this, the present invention aims to propose a target detection method, device, vehicle controller and readable storage medium to solve the problem of poor target detection accuracy in the prior art.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] In a first aspect, the present invention provides a target detection method, the method comprising:

[0008] Extract features from the voxel space determined based on the point cloud data to obtain three-dimensional features of the space;

[0009] Extract features from the two-dimensional image to obtain two-dimensional features to be fused;

[0010] Performing feature extraction on the spatial three-dimensional features and the two-dimensional features to be fused based on the three-dimensional backbone network to obtain target extraction features output by the three-dimensional backbone network;

[0011] Target detection is performed on the target extraction features based on the region of interest pooling network, and the pooling results generated during the target detection process are fused with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network.

[0012] Optionally, the extracting the spatial three-dimensional features and the two-dimensional features to be fused based on the three-dimensional backbone network to obtain target extraction features output by the three-dimensional backbone network includes:

[0013] Fusing the spatial three-dimensional features with the two-dimensional features to be fused to obtain target spatial three-dimensional features;

[0014] The three-dimensional features of the target space are extracted based on the three-dimensional backbone network, and the intermediate features generated in the feature extraction process are fused with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network.

[0015] Optionally, the fusing the spatial three-dimensional feature with the two-dimensional feature to be fused to obtain the target spatial three-dimensional feature includes:

[0016] Acquire a mapping matrix between the point cloud data and the two-dimensional image;

[0017] Determine, according to the mapping matrix, first feature elements corresponding to respective spatial three-dimensional feature elements in the two-dimensional feature to be fused;

[0018] Each spatial three-dimensional feature element in the spatial three-dimensional feature is fused with a first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0019] Optionally, the fusing each spatial three-dimensional feature element in the spatial three-dimensional feature with a first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature includes:

[0020] Determine the feature element to be fused corresponding to the spatial three-dimensional feature element based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional feature to be fused;

[0021] Each spatial three-dimensional feature element in the spatial three-dimensional feature is fused with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0022] Optionally, the extracting three-dimensional features of the target space based on the three-dimensional backbone network, and fusing the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network, includes:

[0023] For at least one intermediate feature generated when the three-dimensional backbone network extracts the three-dimensional features of the target space, fusing the intermediate feature with the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature;

[0024] Input the intermediate fusion features into the next layer of network structure;

[0025] The target extraction features output by the three-dimensional backbone network are obtained.

[0026] Optionally, the fusing the intermediate feature with the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature includes:

[0027] Performing upsampling processing on the intermediate features to obtain upsampled intermediate features having a feature resolution that is the same as a feature resolution of the spatial three-dimensional features;

[0028] Determine, according to a mapping matrix between the point cloud data and the two-dimensional image, second feature elements corresponding to each of the up-sampled intermediate feature elements in the up-sampled intermediate feature and in the two-dimensional feature to be fused;

[0029] fusing each upsampled intermediate feature element in the upsampled intermediate feature with a second feature element corresponding to the upsampled intermediate feature element to obtain a fused intermediate feature;

[0030] Down-sampling is performed on the fused intermediate features to obtain the intermediate fused features having a feature resolution that is the same as the feature resolution of the intermediate features.

[0031] Optionally, the region of interest pooling network includes a first region of interest pooling subnetwork, the pooling result includes a first pooling result, and the target detection is performed on the target extraction feature based on the region of interest pooling network, and the pooling result generated in the target detection process is fused with the two-dimensional feature to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network, including:

[0032] Inputting the target extraction feature into the first pooling layer of the first region of interest pooling subnetwork to obtain the first pooling result output by the first pooling layer;

[0033] Performing feature fusion on the first pooling result and the two-dimensional features to be fused to obtain a first target pooling result;

[0034] The first target pooling result is input into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer.

[0035] Optionally, the region of interest pooling network includes a second region of interest pooling subnetwork, the pooling result includes a second pooling result, and inputting the first target pooling result into a first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer includes:

[0036] Inputting the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain an intermediate target detection result output by the first detection layer;

[0037] Inputting the intermediate target detection result into the second pooling layer of the second region of interest pooling subnetwork to obtain the second pooling result output by the second pooling layer;

[0038] Fusing the second pooling result with the two-dimensional features to be fused to obtain a second target pooling result;

[0039] The second target pooling result is input into the second detection layer of the second region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the second detection layer.

[0040] Optionally, extracting features from the two-dimensional image to obtain two-dimensional features to be fused includes:

[0041] Extracting features from the two-dimensional image based on an image feature encoding network to obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image; wherein the two-dimensional image features include target size features, deviation correction features, and detection frame position features output by the image feature encoding network, and at least one of temporary features generated during feature detection of the two-dimensional image by the image feature encoding network;

[0042] Based on the two-dimensional image features and the semantic segmentation mask features, two-dimensional features to be fused corresponding to the two-dimensional image are determined.

[0043] Optionally, determining the two-dimensional features to be fused corresponding to the two-dimensional image based on the two-dimensional image features and the semantic segmentation mask features includes:

[0044] Performing upsampling processing on the two-dimensional image features to obtain upsampled two-dimensional image features having a feature resolution that is the same as a feature resolution of the semantic segmentation mask features;

[0045] The upsampled two-dimensional image features and the semantic segmentation mask features are subjected to feature fusion processing to obtain the two-dimensional features to be fused.

[0046] Optionally, the method further comprises:

[0047] Inputting the sample two-dimensional image into an initial feature encoding network to obtain sample two-dimensional image features and sample semantic segmentation mask features output by the initial feature encoding network;

[0048] Inputting the sample two-dimensional image into a semantic segmentation model to obtain a reference semantic segmentation mask feature output by the semantic segmentation model;

[0049] Based on the reference semantic segmentation mask features and the label two-dimensional image features corresponding to the sample two-dimensional image features, the initial feature encoding network is trained to obtain the image feature encoding network.

[0050] In a second aspect, the present invention provides a target detection device, the device comprising:

[0051] A spatial three-dimensional feature module is used to extract features from the voxel space determined based on the point cloud data to obtain spatial three-dimensional features;

[0052] The feature module to be fused is used to extract features from the two-dimensional image to obtain two-dimensional features to be fused;

[0053] A feature extraction module, used for extracting the spatial three-dimensional features and the two-dimensional features to be fused based on a three-dimensional backbone network to obtain target extracted features output by the three-dimensional backbone network;

[0054] The pooling result fusion module is used to perform target detection on the target extraction features based on the region of interest pooling network, and to fuse the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain the three-dimensional target detection results output by the region of interest pooling network.

[0055] Optionally, the feature extraction module includes:

[0056] A spatial three-dimensional feature fusion submodule, used to fuse the spatial three-dimensional feature with the two-dimensional feature to be fused to obtain a target spatial three-dimensional feature;

[0057] The target extraction feature submodule is used to extract the three-dimensional features of the target space based on the three-dimensional backbone network, and fuse the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network.

[0058] Optionally, the spatial three-dimensional feature fusion submodule includes:

[0059] A mapping matrix submodule, used to obtain a mapping matrix between the point cloud data and the two-dimensional image;

[0060] A first feature element submodule, used to determine, according to the mapping matrix, first feature elements respectively corresponding to each of the spatial three-dimensional feature elements in the two-dimensional feature to be fused;

[0061] The spatial three-dimensional feature element fusion submodule is used to fuse each spatial three-dimensional feature element in the spatial three-dimensional feature with the first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0062] Optionally, the spatial three-dimensional feature element fusion submodule includes:

[0063] The feature element to be fused submodule is used to determine the feature element to be fused corresponding to the spatial three-dimensional feature element based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional feature to be fused;

[0064] The target spatial three-dimensional feature generation submodule is used to fuse each spatial three-dimensional feature element in the spatial three-dimensional feature with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0065] Optionally, the target extraction feature submodule includes:

[0066] An intermediate fusion feature submodule, for fusing at least one intermediate feature generated when the three-dimensional backbone network extracts the three-dimensional features of the target space with the two-dimensional features to be fused, to obtain an intermediate fusion feature corresponding to the intermediate feature;

[0067] An intermediate fusion feature input submodule, used to input the intermediate fusion feature into the next layer of network structure;

[0068] The target extraction feature acquisition submodule is used to obtain the target extraction feature output by the three-dimensional backbone network.

[0069] Optionally, the intermediate fusion feature submodule includes:

[0070] An upsampling intermediate feature submodule, used to perform upsampling processing on the intermediate feature to obtain an upsampled intermediate feature with the same feature resolution as the spatial three-dimensional feature;

[0071] A second feature element submodule, for determining, according to a mapping matrix between the point cloud data and the two-dimensional image, second feature elements corresponding to each of the up-sampled intermediate feature elements in the up-sampled intermediate feature and in the two-dimensional feature to be fused;

[0072] A fusion intermediate feature submodule, used for fusing each upsampled intermediate feature element in the upsampled intermediate feature with a second feature element corresponding to the upsampled intermediate feature element to obtain a fused intermediate feature;

[0073] The intermediate fusion feature generation submodule is used to downsample the fused intermediate features to obtain the intermediate fusion features with the same feature resolution as the intermediate features.

[0074] Optionally, the region of interest pooling network includes a first region of interest pooling subnetwork, the pooling result includes a first pooling result, and the pooling result fusion module includes:

[0075] A first pooling result submodule, used for inputting the target extraction feature into the first pooling layer of the first region of interest pooling subnetwork, to obtain the first pooling result output by the first pooling layer;

[0076] A first target pooling result submodule, used for performing feature fusion on the first pooling result and the two-dimensional features to be fused to obtain a first target pooling result;

[0077] The three-dimensional target detection result submodule is used to input the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer.

[0078] Optionally, the region of interest pooling network includes a second region of interest pooling subnetwork, the pooling result includes a second pooling result, and the three-dimensional object detection result submodule includes:

[0079] An intermediate target detection result submodule, used for inputting the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain the intermediate target detection result output by the first detection layer;

[0080] A second pooling result submodule, used for inputting the intermediate target detection result into the second pooling layer of the second region of interest pooling subnetwork, to obtain the second pooling result output by the second pooling layer;

[0081] A second target pooling result acquisition submodule, used to fuse the second pooling result with the two-dimensional features to be fused to obtain a second target pooling result;

[0082] The second target pooling result is input into the second detection layer of the second region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the second detection layer.

[0083] Optionally, the feature modules to be fused include:

[0084] A feature extraction submodule, configured to extract features from the two-dimensional image based on an image feature encoding network, and obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image; wherein the two-dimensional image features include target size features, deviation correction features, and detection frame position features output by the image feature encoding network, and at least one of temporary features generated during feature detection of the two-dimensional image by the image feature encoding network;

[0085] The two-dimensional feature to be fused submodule is used to determine the two-dimensional feature to be fused corresponding to the two-dimensional image based on the two-dimensional image feature and the semantic segmentation mask feature.

[0086] Optionally, the two-dimensional feature submodule to be fused includes:

[0087] An up-sampling two-dimensional image feature submodule, used to perform up-sampling processing on the two-dimensional image feature to obtain an up-sampled two-dimensional image feature having a feature resolution that is the same as the feature resolution of the semantic segmentation mask feature;

[0088] The two-dimensional feature to be fused generating submodule is used to perform feature fusion processing on the up-sampled two-dimensional image feature and the semantic segmentation mask feature to obtain the two-dimensional feature to be fused.

[0089] Optionally, the device further comprises:

[0090] An initial feature coding network module, used for inputting a sample two-dimensional image into the initial feature coding network, and obtaining sample two-dimensional image features and sample semantic segmentation mask features output by the initial feature coding network;

[0091] A reference semantic segmentation mask feature module is used to input the sample two-dimensional image into a semantic segmentation model to obtain a reference semantic segmentation mask feature output by the semantic segmentation model;

[0092] The image feature encoding network training module is used to train the initial feature encoding network based on the reference semantic segmentation mask features and the label two-dimensional image features corresponding to the sample two-dimensional image features to obtain the image feature encoding network.

[0093] In a third aspect, the present invention provides a vehicle controller, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned target detection method when executing the computer program.

[0094] In a fourth aspect, the present invention provides a readable storage medium, when the instructions in the readable storage medium are executed by a processor of a vehicle controller, the vehicle controller is enabled to execute the above-mentioned target detection method.

[0095] In a fifth aspect, the present invention provides a vehicle, comprising the above-mentioned vehicle controller.

[0096] Compared with the prior art, the target detection method, device, vehicle controller and readable storage medium described in the present invention have the following advantages:

[0097] In summary, an embodiment of the present invention provides a target detection method, including: extracting features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features; extracting features from a two-dimensional image to obtain two-dimensional features to be fused; wherein the two-dimensional image and the point cloud data are captured in the same time and space; extracting features from the spatial three-dimensional features and the two-dimensional features to be fused based on a three-dimensional backbone network to obtain target extraction features output by the three-dimensional backbone network; performing target detection on the target extraction features based on a region of interest pooling network, and fusing the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network. In an embodiment of the present invention, in the process of performing three-dimensional target detection based on point cloud data, the two-dimensional features to be fused corresponding to the two-dimensional image can be fused, so that in the process of performing three-dimensional target detection, some information unique to the two-dimensional image (such as texture information, etc.) can be considered, and the accuracy of performing three-dimensional target detection can be improved through the complementarity of point cloud information and two-dimensional images, thereby obtaining a high-precision three-dimensional detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0099] Figure 1 A flowchart of a target detection method provided by an embodiment of the present invention;

[0100] Figure 2 A schematic diagram of a three-dimensional backbone network provided by an embodiment of the present invention;

[0101] Figure 3 A schematic diagram of a region of interest pooling network provided by an embodiment of the present invention;

[0102] Figure 4 A flowchart of another target detection method provided by an embodiment of the present invention;

[0103] Figure 5 A schematic diagram of an image feature coding network training provided by an embodiment of the present invention;

[0104] Figure 6 A schematic diagram of a target detection network provided by an embodiment of the present invention;

[0105] Figure 7 Another target detection network schematic diagram provided by an embodiment of the present invention;

[0106] Figure 8 A structural block diagram of a target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0107] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0108] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0109] Reference Figure 1 , shows a step flow chart of a target detection method provided by an embodiment of the present invention.

[0110] Step 101 , extracting features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features.

[0111] The collection of point data on the surface of an object obtained by a measuring instrument is called point cloud data. For example, laser radar, 3D laser scanner, or photographic scanner can be used to obtain relatively dense point cloud data.

[0112] Taking the application of point cloud data in the field of autonomous driving as an example, in order to detect targets around the vehicle, a laser radar can be installed on the vehicle, and the laser radar can emit dense laser beams to the surrounding environment, and then generate point cloud data of the surrounding environment based on the reflection of the laser beam on the object. It should be noted that in the embodiment of the present invention, the method of obtaining point cloud data through the laser radar is not specifically limited. For example, the point cloud data of the environment in front of the vehicle can be obtained by the front laser radar of the vehicle, and the panoramic point cloud data around the vehicle can be generated by multiple laser radars arranged around the vehicle. The panoramic point cloud data around the vehicle can also be obtained by a rotating laser radar arranged on the roof. Similarly, in the embodiment of the present invention, the method of obtaining point cloud data is not specifically limited. For example, the point cloud data of the surrounding environment of the vehicle can be obtained by the above-mentioned laser radar, and the distance and direction between multiple points in the surrounding environment and the vehicle can be calculated by a binocular stereo vision solution, thereby generating point cloud data of the surrounding environment of the vehicle.

[0113] Since each point in the point cloud data contains orientation information relative to the point cloud shooting party (such as a vehicle), after obtaining the point cloud data, each point in the point cloud data can be projected into a space with the point cloud shooting party as the origin to obtain a point cloud data space.

[0114] In order to facilitate the feature extraction and classification of point cloud features in the point cloud data space, the point cloud data space can be segmented to obtain multiple smaller volume elements, so that the three-dimensional points in the same volume element can be classified as a whole to improve the classification efficiency. Among them, the volume element (Volume Pixel, Voxel) can be referred to as voxel, which can be expressed by stereo rendering or extracting polygonal isosurfaces with given threshold contours.

[0115] Specifically, the point cloud data can be divided into equal volumes based on a set of length, width and height data to obtain multiple voxels of the same volume. The technicians can also use other methods to divide the voxels, which is not specifically limited in the embodiment of the present invention.

[0116] In the embodiment of the present invention, the artificial neural network (ANN), convolutional neural network (CNN) layer and other technologies can be used to extract features from the voxel space to obtain the spatial three-dimensional features corresponding to the voxel space. It should be noted that since there may be a large number of empty voxels (voxels that do not contain any points) in the voxel, in order to improve the operation efficiency and save the amount of operation, only the non-empty voxels can be feature extracted.

[0117] Artificial Neural Network (ANN), ANN is a technology that imitates biological neural networks. It connects multiple eigenvalues ​​and combines linear and nonlinearity to finally achieve the set goal. It includes an input layer, an output layer, and at least one hidden layer in the middle. Convolutional Neural Networks (CNN) is a type of feedforward neural network (FNN) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. The spatial three-dimensional features of voxels can be extracted through the pooling layer of the convolutional neural network. Technicians can select a suitable spatial three-dimensional feature extraction method as needed, and the embodiments of the present invention do not specifically limit this.

[0118] Step 102: extract features from the two-dimensional image to obtain two-dimensional features to be fused.

[0119] In the embodiment of the present invention, a two-dimensional image corresponding to the point cloud data can also be obtained. The point cloud data and the two-dimensional image have a corresponding relationship. The point cloud data and the two-dimensional image are shot at the same time and space (i.e., the same time and space), and contain the same scene. For example, a laser radar sensor and a visible light camera can be installed on a vehicle, and the environment around the vehicle can be shot simultaneously by the laser radar sensor and the visible light camera to obtain point cloud data and corresponding two-dimensional images shot at the same time and space.

[0120] After obtaining the two-dimensional image, a neural network algorithm can be used to extract features from the two-dimensional image to obtain two-dimensional features to be fused corresponding to the two-dimensional image. The two-dimensional features to be fused may include one or more of any features that can be extracted based on the two-dimensional image. For example, semantic segmentation mask features, target size features, deviation correction features, detection frame position features, etc.

[0121] Step 103, extracting the three-dimensional features of the space based on the three-dimensional backbone network, and fusing the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain the target extracted features output by the three-dimensional backbone network.

[0122] In an embodiment of the present invention, a three-dimensional backbone network (3D backbone) can be used to extract spatial three-dimensional features, and during the processing of the three-dimensional backbone network, the intermediate features generated in the three-dimensional backbone network are fused with the two-dimensional features to be fused, thereby improving the accuracy of subsequent three-dimensional target detection based on the output of the three-dimensional backbone network.

[0123] Specifically, since the three-dimensional backbone network includes multiple three-dimensional feature extraction layers, multiple intermediate features will be generated during the process of the three-dimensional backbone network processing the spatial three-dimensional features, and the intermediate features generated by one three-dimensional feature extraction layer will enter the next three-dimensional feature extraction layer for further extraction. The intermediate features generated during the processing of the three-dimensional backbone network can be fused with the above-mentioned two-dimensional features to be fused, and the feature extraction process of the three-dimensional backbone network can be continued according to the fusion results corresponding to the intermediate features, so as to obtain the target extraction features that are finally output by the three-dimensional backbone network and fuse the information of the two-dimensional image.

[0124] Reference Figure 2 , Figure 2 A schematic diagram of a three-dimensional backbone network provided by an embodiment of the present invention is shown. Figure 2 As shown, the three-dimensional backbone network 20 includes three layers of three-dimensional feature extraction layers 21 to 23. After the spatial three-dimensional features are input into the three-dimensional backbone network 20, the three-dimensional feature extraction layers 21 and 22 can both output corresponding intermediate features. The intermediate features output by the three-dimensional feature extraction layer 21 can be fused with the two-dimensional features to be fused, and the fused intermediate features are input into the three-dimensional feature extraction layer 22 for subsequent processing; the intermediate features output by the three-dimensional feature extraction layer 22 can also be fused with the two-dimensional features to be fused, and the fused intermediate features are input into the three-dimensional feature extraction layer 23 for subsequent processing; the intermediate features output by the three-dimensional feature extraction layer 21 can also be fused with the two-dimensional features to be fused, and the fused intermediate features are input into the three-dimensional feature extraction layer 22, and then the intermediate features output by the three-dimensional feature extraction layer 22 can be fused with the two-dimensional features to be fused, and the fused intermediate features are input into the three-dimensional feature extraction layer 23.

[0125] Step 104, performing target detection on the target extraction features based on the region of interest pooling network, and fusing the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network.

[0126] In an embodiment of the present invention, target detection can be performed on target extraction features through a region of interest pooling network (RoI Pooling Network), thereby obtaining a three-dimensional target detection result. The above two-dimensional features to be fused can be fused into the process of processing target extraction features by the region of interest pooling network, thereby obtaining a more accurate three-dimensional target detection result.

[0127] Specifically, since the region of interest pooling network is a multi-layer network structure, the features output by any layer of the network structure in the region of interest pooling network can be fused with the two-dimensional features to be fused during the process of processing the target extraction features in the region of interest pooling network, and the fused results are input to the next layer of the network structure, and the three-dimensional target detection results output by the region of interest pooling network are obtained. Among them, based on the different settings of the detection head in the region of interest pooling network, the three-dimensional target detection results may also be different. For example, the three-dimensional target detection results may include the position, distance, detection frame, etc. of pedestrians, and may also include the position, distance, detection frame, etc. of other vehicles. The embodiment of the present invention does not limit the specific content of the three-dimensional target detection results.

[0128] Reference Figure 3 , Figure 3 FIG. 4 shows a schematic diagram of a region of interest pooling network provided by an embodiment of the present invention, such as Figure 3 As shown, the region of interest pooling network 30 may include a pooling layer 31 and a detection layer 32 composed of a detection head. After the target extraction features are input into the region of interest pooling network, the pooling result 33 output by the pooling layer 31 may be fused with the two-dimensional features to be fused, and the fused result may be input into the subsequent detection layer 32 for subsequent processing, and finally a three-dimensional target detection result output by the region of interest pooling network 30 is obtained.

[0129] In summary, an embodiment of the present invention provides a target detection method, including: extracting features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features; extracting features from a two-dimensional image to obtain two-dimensional features to be fused; wherein the two-dimensional image and the point cloud data are captured at the same time and space; extracting features from the spatial three-dimensional features and the two-dimensional features to be fused to obtain target extraction features output by a three-dimensional backbone network; performing target detection on the target extraction features based on a region of interest pooling network, and fusing the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network. In an embodiment of the present invention, in the process of performing three-dimensional target detection based on point cloud data, the two-dimensional features to be fused corresponding to the two-dimensional image can be fused, so that in the process of performing three-dimensional target detection, some information unique to the two-dimensional image (such as texture information, etc.) can be considered, and the accuracy of three-dimensional target detection can be improved through the complementarity of point cloud information and two-dimensional images, thereby obtaining a high-precision three-dimensional detection result.

[0130] Reference Figure 4 , Figure 4 A flowchart of another target detection method provided by an embodiment of the present invention is shown.

[0131] Step 201 , extracting features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features.

[0132] This step can refer to the above step 101 and will not be described again here.

[0133] Optionally, the spatial three-dimensional features corresponding to the voxel space can be determined by the exemplary method shown in the following sub-steps 2011 to sub-step 2016. The technician can also determine the spatial three-dimensional features in other ways. For example, the point cloud data can be directly input into the spatial three-dimensional feature extraction model to obtain the spatial three-dimensional features corresponding to the voxel space of the point cloud data. The embodiment of the present invention does not make specific limitations.

[0134] Sub-step 2011, segmenting the point cloud image into a voxel space including a plurality of voxels; wherein each voxel includes the same number of points.

[0135] In an embodiment of the present invention, after obtaining the point cloud data, the point cloud data can be segmented to convert the disordered points in the point cloud into ordered voxels, and obtain the voxel space corresponding to the point cloud data. Specifically, the point cloud data can be divided into a plurality of grids with the same length, width and height, and each grid is used as a voxel, so as to obtain a voxel space composed of all voxels.

[0136] Sub-step 2012, obtaining the spatial coordinates and reflectivity corresponding to each point in the point cloud image.

[0137] For each point in the voxel, it contains spatial coordinate information and reflectivity information, where the spatial coordinates can be represented by x, y, and z, and the reflectivity can be represented by r. Then each point corresponds to description information (x, y, z, r).

[0138] Sub-step 2013, based on the spatial coordinates of each point in the voxel, determine the centroid coordinates corresponding to the voxel.

[0139] The centroid coordinates of each voxel can also be calculated according to the spatial coordinates of each point in each voxel. The centroid coordinates of the voxel can be expressed as (Vx, Vy, Vz).

[0140] Sub-step 2014, determining the point vector corresponding to each point in the point cloud image according to the centroid coordinates, the spatial coordinates and the reflectivity.

[0141] After obtaining the centroid coordinates, the centroid coordinates of the voxel can be combined with the description information of each point in the voxel (including the spatial coordinates and reflectivity of the point) to obtain the point vector of the point in the voxel.

[0142] For example, if the centroid coordinates of a voxel can be expressed as (Vx, Vy, Vz), and the description information of a point in the voxel is (x, y, z, r), then the point vector of the point can be described by a feature vector (x, y, z, r, Vx, Vy, Vz) with a feature value of 7.

[0143] Sub-step 2015, determining the voxel vector corresponding to the voxel based on the point vector corresponding to each point in the voxel.

[0144] After obtaining the point vector of each point, the point vector can be input into a fully connected network consisting of a linear layer, a batch layer, and a Relu layer to further extract features of each point and obtain the first point vector corresponding to each point, where the first point vector can be represented by a feature vector with a length of 16.

[0145] Next, the first point vector is subjected to maximum pooling to obtain a second point vector, and the second point vector corresponding to each point in the voxel is combined with the first point vector and passed through at least one VFE layer to obtain a voxel vector corresponding to the voxel.

[0146] Sub-step 2016, based on the voxel vectors corresponding to the respective voxels, determining the spatial three-dimensional feature map corresponding to the voxel space.

[0147] In the embodiment of the present invention, the voxel vectors of all voxels may be combined to obtain an initial spatial three-dimensional feature map corresponding to the voxel space.

[0148] You can also set three 3D convolutional layers Conv3D (cin, cout, k, s, p) to further process the initial spatial three-dimensional feature map to obtain a spatial three-dimensional feature map corresponding to the voxel space. Cin and cout in the convolutional layer represent the feature length, and k, s, and p represent the convolution kernel size.

[0149] For example, the first 3D convolution layer can be set to Con3D(128, 64, 3, (2, 1, 1), (1, 1, 1)), the second 3D convolution layer can be set to Con3D(64, 64, 3, (1, 1, 1), (0, 1, 1)), and the third 3D convolution layer can be set to Con3D(64, 64, 3, (2, 1, 1), (1, 1, 1)).

[0150] Step 202: extract features from the two-dimensional image based on an image feature encoding network to obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image.

[0151] In an embodiment of the present invention, feature extraction can be performed on a two-dimensional image based on a pre-trained image feature encoding network to obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image, wherein the semantic segmentation mask features can also be called semantic features, and the process of obtaining the above-mentioned semantic segmentation mask features can also be called semantic segmentation.

[0152] Specifically, the image feature encoding network can be established based on the feature aggregation network (Deep Layer Aggregation, DLA). For example, the image feature encoding network can adopt the DLA-34 network in the central network (CenterNet). At the same time, iterative deep aggregation, hierarchical deep aggregation and variable convolution can also be applied in the image feature encoding network to achieve the image pixel feature fusion between the high-level network and the low-level network in the image feature encoding network, so as to obtain the two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image.

[0153] It should be noted that the two-dimensional image features include but are not limited to the target size features, deviation correction features, detection frame position features output by the image feature encoding network, and at least one of the temporary features generated during the feature detection process of the image feature encoding network on the two-dimensional image. Among them, the above-mentioned target size features, deviation correction features, and detection frame position features can be the features finally output by the image feature encoding network, and the above-mentioned temporary features can represent the features output by a certain network layer in the image feature encoding network. The temporary features can be used to obtain the more original features of the two-dimensional image, thereby providing richer feature information for subsequent target detection tasks.

[0154] Optionally, in an embodiment of the present invention, the above-mentioned image feature encoding network can be pre-trained based on the following method:

[0155] The sample two-dimensional image is input into the initial feature encoding network to obtain the sample two-dimensional image features and sample semantic segmentation mask features output by the initial feature encoding network; the sample two-dimensional image is input into the semantic segmentation model to obtain the reference semantic segmentation mask features output by the semantic segmentation model; based on the reference semantic segmentation mask features and the label two-dimensional image features corresponding to the sample two-dimensional image features, the initial feature encoding network is trained to obtain the image feature encoding network. Through the above training process, an image feature encoding network that can simultaneously output two-dimensional image features and sample semantic segmentation mask features can be obtained, so that the features of the two-dimensional image can be extracted through an image feature encoding network with a relatively simple structure, and there is no need to introduce a semantic segmentation model in the subsequent three-dimensional target detection process to obtain the semantic segmentation mask features of the two-dimensional image, which reduces the computational overhead and improves the computational efficiency.

[0156] Reference Figure 5 , Figure 5FIG. 4 shows a schematic diagram of an image feature coding network training provided by an embodiment of the present invention. Figure 5 As shown, the sample two-dimensional image can be input into the initial feature encoding network 51 to obtain the sample target size feature, sample deviation correction feature, sample detection box position feature and sample semantic segmentation mask feature output by the initial feature encoding network 51; the sample two-dimensional image can be simultaneously input into the semantic segmentation model 52 to obtain the reference semantic segmentation mask feature output by the semantic segmentation model 52; and based on the above-mentioned reference semantic segmentation mask feature, as well as the label target size feature, label deviation correction feature and label detection box position feature corresponding to the sample two-dimensional image feature, the initial feature encoding network is trained to obtain the image feature encoding network.

[0157] It should be noted that the initial feature encoding network can be composed of multiple two-dimensional feature extraction layers. For example, the structure of the initial feature encoding network can be as described above. Figure 5 As shown, it includes multiple two-dimensional feature extraction layers 54, wherein the numbers in the feature extraction layers 54 represent the feature dimensions extracted by the feature extraction layers, the solid arrows between the feature extraction layers 54 represent the feature extraction directions, and the dotted arrows between the feature extraction layers 54 represent the feature fusion directions.

[0158] The above-mentioned semantic segmentation model may include but is not limited to SegFormer, Global Convolutional Network (GCN), Discriminative Feature Network (DFN), PSPNet (Pyramid Scene Parsing Network), DeepLab and other models, and the embodiment of the present invention does not make specific limitations.

[0159] Optionally, during the training process of the image feature coding network, the sample two-dimensional image can also be processed by a data enhancement module to generate more samples based on the sample two-dimensional image, thereby increasing the sample size of the training image feature coding network and enhancing the training effect of the image feature coding network. The above-mentioned data enhancement module may include but is not limited to a cut&paste data enhancement module, etc. The cut&paste data enhancement module may be a semantic-based cut&paste data enhancement module or a common cut&paste data enhancement module, which is not specifically limited in the embodiment of the present invention.

[0160] like Figure 5 As shown, the sample two-dimensional image can be first input into the data enhancement module 55, which performs differentiation processing on the sample two-dimensional image to increase the number of sample two-dimensional images, and then the differentiated sample two-dimensional image is input into the semantic segmentation model 52 and the initial feature encoding network 51 to train the image feature encoding network.

[0161] Step 203: Determine the two-dimensional features to be fused corresponding to the two-dimensional image based on the two-dimensional image features and the semantic segmentation mask features.

[0162] Reference Figure 6 , Figure 6 A schematic diagram of a target detection network provided by an embodiment of the present invention is shown. Figure 6 As shown, the target detection network may include an image feature encoding network 61, which includes a network layer 68 of an encoding network. The image feature encoding network 61 inputs a two-dimensional image and outputs a two-dimensional image feature 62 and a semantic segmentation mask feature, wherein the two-dimensional image feature includes Figure 6 The target size features, deviation correction features, and detection frame position features shown can fuse the above-mentioned semantic segmentation mask features, two-dimensional image features 62, and temporary features output by the network layer 68 of the encoding network to obtain a two-dimensional feature to be fused 63.

[0163] In an embodiment of the present invention, each feature in the two-dimensional image feature corresponding to the two-dimensional image and the above-mentioned semantic segmentation mask feature can be fused to obtain a two-dimensional feature to be fused. Specifically, the above-mentioned features can be directly fused to obtain a two-dimensional feature to be fused. Considering that the feature resolutions of these features may be different, in order to improve the fusion effect, the above-mentioned features can also be fused in the following manner from sub-step 2031 to sub-step 2032:

[0164] Optionally, step 203 may include:

[0165] Sub-step 2031 , upsampling the two-dimensional image features to obtain up-sampled two-dimensional image features having the same feature resolution as the feature resolution of the semantic segmentation mask features.

[0166] In an embodiment of the present invention, the two-dimensional image features can be upsampled to obtain upsampled two-dimensional image features with the same feature resolution as the semantic segmentation mask features, so that the features involved in the fusion maintain a consistent feature resolution, thereby improving the fusion effect and obtaining more accurate two-dimensional features to be fused.

[0167] Sub-step 2032, performing feature fusion processing on the up-sampled two-dimensional image features and the semantic segmentation mask features to obtain the two-dimensional features to be fused.

[0168] For example, if the two-dimensional image features corresponding to a two-dimensional image include target size features and detection frame position features, the target size features and the detection frame position features can be upsampled to obtain upsampled target size features and upsampled detection frame position features. The upsampled target size features and the detection frame position features have the same feature resolution as the semantic segmentation mask features of the above two-dimensional image. The upsampled target size features, the detection frame position features and the semantic segmentation mask features with the same feature resolution are then feature fused to obtain the two-dimensional features to be fused corresponding to the two-dimensional image.

[0169] Step 204: fuse the spatial three-dimensional features with the two-dimensional features to be fused to obtain target spatial three-dimensional features.

[0170] In an embodiment of the present invention, before further feature extraction is performed on the spatial three-dimensional features corresponding to the point cloud data, the spatial three-dimensional features can be fused with the two-dimensional features to be fused first, and then the target spatial three-dimensional features obtained after the fusion are used as the input of the subsequent three-dimensional target detection process, thereby further improving the effect of three-dimensional target detection.

[0171] Optionally, since the spatial three-dimensional feature is determined based on the spatial point cloud and is a three-dimensional feature, and the two-dimensional feature to be fused is determined based on the two-dimensional image and is a two-dimensional feature, it is necessary to fuse the spatial three-dimensional feature and the two-dimensional feature to be fused according to the correspondence between the feature elements in the spatial three-dimensional feature and the feature elements in the two-dimensional feature to be fused. The specific process can be seen from the following sub-steps 2041 to 2042:

[0172] Sub-step 2041, obtaining a mapping matrix between the point cloud data and the two-dimensional image.

[0173] In order to determine the corresponding relationship between the spatial three-dimensional features and each feature element in the two-dimensional features to be fused, the mapping matrix between the point cloud data and the two-dimensional image can be obtained first.

[0174] Specifically, the mapping matrix is ​​used to represent the correspondence between points in the point cloud data and pixels in the two-dimensional image. For example, if the mapping matrix between the point cloud data and the two-dimensional image is M, and the position of a point in the point cloud data is p(x, y, z), the corresponding pixel position in the two-dimensional image is p′(x′, y ′ ), then the relationship between the above two points can be expressed as: p′=M×p.

[0175] The mapping matrix can be determined in the process of calibrating the laser radar and the visual camera and stored in the memory, and the mapping matrix can be obtained from the memory when used. The technician can also use other methods to determine the above mapping matrix, which is not specifically limited in the embodiment of the present invention.

[0176] Sub-step 2042, determining, according to the mapping matrix, first feature elements in the two-dimensional features to be fused that correspond to the respective three-dimensional feature elements in the three-dimensional features.

[0177] In the embodiment of the present invention, the target pixel point corresponding to each voxel in the voxel space in the two-dimensional image may be determined according to the mapping matrix.

[0178] Specifically, the voxel center of the voxel can be mapped to the two-dimensional image through the above-mentioned mapping matrix to obtain the mapping position of the voxel center in the two-dimensional image, and establish a correspondence between the pixel point at the mapping position and the voxel, thereby establishing a first correspondence between each voxel and the pixel point in the two-dimensional image. The technician can also use other methods to establish the first correspondence between the voxel and the pixel point, which is not specifically limited in the embodiment of the present invention.

[0179] Since the three-dimensional feature elements in the spatial three-dimensional features have a corresponding relationship with the voxels, and the two-dimensional feature elements of the two-dimensional features to be fused have a corresponding relationship with the pixels in the two-dimensional image, according to the above two sets of corresponding relationships, combined with the target pixel points corresponding to each spatial three-dimensional feature element, the first feature elements corresponding to each spatial three-dimensional feature element in the two-dimensional features to be fused can be determined.

[0180] Sub-step 2043, fusing each spatial three-dimensional feature element in the spatial three-dimensional feature with the first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0181] In the embodiment of the present invention, for each spatial three-dimensional feature element in the spatial three-dimensional feature, it can be fused with the corresponding first feature element to obtain the target spatial three-dimensional feature.

[0182] Furthermore, since the real scene information represented by the spatial three-dimensional feature element is not strictly consistent with the real scene information represented by the first feature element corresponding to the spatial three-dimensional feature element, in an embodiment of the present invention, in order to improve the accuracy of the fused target spatial three-dimensional features, the above fusion process can also be performed based on a bilinear interpolation method or an adjacent feature aggregation method. For details, please refer to the following sub-steps A1 to A2:

[0183] Optionally, sub-step 2043 may include:

[0184] Sub-step A1, based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional features to be fused, determine the feature element to be fused corresponding to the spatial three-dimensional feature element.

[0185] In the embodiment of the present invention, the feature element to be fused corresponding to the spatial three-dimensional feature element may be determined based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional features to be fused.

[0186] Specifically, for a first feature element corresponding to a spatial three-dimensional feature element, a preset number of feature elements closest to the first feature element in the two-dimensional features to be fused can be determined as adjacent feature elements of the first feature element, and then the first feature element and the adjacent feature elements of the first feature element are fused into the feature element to be fused corresponding to the spatial three-dimensional feature element. The above preset number can be flexibly set based on actual needs and is not specifically limited in the embodiments of the present invention.

[0187] Furthermore, the feature elements to be fused may be determined based on a bilinear difference method. Specifically, the feature elements to be fused may be determined based on the following formula 1:

[0188] V (p) = k(F (N(p′)) ) Formula 1

[0189] Among them, V (p) represents the feature elements to be fused, F (N(p′)) represents the set of the first feature element and the adjacent feature elements, and k is a bilinear interpolation function. Based on the above formula, the feature elements to be fused corresponding to each spatial three-dimensional feature element can be determined.

[0190] Sub-step A2, fusing each spatial three-dimensional feature element in the spatial three-dimensional feature with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0191] In an embodiment of the present invention, the feature element to be fused corresponding to the spatial three-dimensional feature element can be determined based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional feature to be fused, and each spatial three-dimensional feature element in the spatial three-dimensional feature is fused with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature, thereby improving the accuracy of the target spatial three-dimensional feature finally obtained, which helps to improve the accuracy in the subsequent target detection process.

[0192] Step 205 , extracting the three-dimensional features of the target space based on the three-dimensional backbone network, and fusing the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network.

[0193] In an embodiment of the present invention, a three-dimensional backbone network (3D backbone) can be used to extract three-dimensional features of the target space, and during the processing of the three-dimensional backbone network, the intermediate features generated in the three-dimensional backbone network are fused with the two-dimensional features to be fused, thereby improving the accuracy of subsequent three-dimensional target detection based on the output of the three-dimensional backbone network.

[0194] Specifically, since the 3D backbone network includes multiple 3D feature extraction layers, multiple intermediate features will be generated during the 3D backbone network's processing of the 3D features of the target space. The intermediate features generated by one 3D feature extraction layer will enter the next 3D feature extraction layer for further extraction. The intermediate features generated during the 3D backbone network processing can be fused with the above-mentioned 2D features to be fused, and the feature extraction process of the 3D backbone network can be continued based on the fusion results corresponding to the intermediate features, thereby obtaining the target extraction features that are finally output by the 3D backbone network and fused with the image features.

[0195] Optionally, step 205 may include:

[0196] In sub-step 2051, for at least one intermediate feature generated when the three-dimensional backbone network extracts the three-dimensional features of the target space, the intermediate feature is fused with the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature.

[0197] In an embodiment of the present invention, for at least one intermediate feature generated when the three-dimensional backbone network extracts three-dimensional features of the target space, feature fusion is performed on the intermediate feature and the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature.

[0198] like Figure 6 As shown, the target detection network may also include a three-dimensional backbone network 65 and a three-dimensional feature extraction layer 66 included in the three-dimensional backbone network 65. The spatial three-dimensional feature 64 may be determined based on the point cloud data first, and then the fusion result of the spatial three-dimensional feature 64 and the two-dimensional feature to be fused 63 is input into the three-dimensional feature extraction layer 66 for feature extraction, and the intermediate features output by at least one three-dimensional feature extraction layer 66 are fused with the two-dimensional feature to be fused 63, and the fused intermediate fusion features are input into the next layer of network structure. For example, the intermediate features output by the last three layers of three-dimensional feature extraction layers in the three-dimensional backbone network 65 may be fused with the two-dimensional feature to be fused 63. It should be noted that the process of converting the point cloud data into the spatial three-dimensional feature 64 may occur outside the three-dimensional backbone network or within the three-dimensional backbone network, and the embodiment of the present invention does not specifically limit this.

[0199] Furthermore, when the three-dimensional backbone network extracts the three-dimensional features of the target space, since the feature resolutions of the intermediate features output by each feature extraction layer in the three-dimensional backbone network are different, and the feature resolution of the intermediate features may be smaller than the feature resolution of the three-dimensional features of the target space, in order to improve the fusion accuracy of the intermediate features and the two-dimensional features to be fused, the intermediate features and the two-dimensional features to be fused can also be fused based on upsampling technology. For details, please refer to the following sub-steps B1 to B4.

[0200] Optionally, sub-step 2051 may include:

[0201] Sub-step B1, performing upsampling processing on the intermediate feature to obtain an upsampled intermediate feature having a feature resolution that is the same as the feature resolution of the spatial three-dimensional feature;

[0202] Sub-step B2, determining, according to a mapping matrix between the point cloud data and the two-dimensional image, second feature elements corresponding to each up-sampled intermediate feature element in the up-sampled intermediate feature and in the two-dimensional feature to be fused.

[0203] This step can refer to the process of fusing the spatial three-dimensional features with the two-dimensional features to be fused in the above step 204, and will not be described in detail in this embodiment of the present invention.

[0204] Sub-step B3, fusing each upsampled intermediate feature element in the upsampled intermediate feature with the second feature element corresponding to the upsampled intermediate feature element to obtain a fused intermediate feature.

[0205] This step can refer to the process of fusing the spatial three-dimensional features with the two-dimensional features to be fused in the above step 204, and will not be described in detail in this embodiment of the present invention.

[0206] Sub-step B4, down-sampling the fused intermediate features to obtain the intermediate fused features having the same feature resolution as the intermediate features.

[0207] In an embodiment of the present invention, by upsampling the intermediate features to have the same feature resolution as the two-dimensional features to be fused, then fusing them with the two-dimensional features to be fused, and downsampling the fused intermediate features to the feature resolution of the intermediate features, it helps to improve the effect of fusing the intermediate features with the two-dimensional features to be fused, and improve the accuracy of the intermediate fused features.

[0208] Sub-step 2052, inputting the intermediate fusion features into the next layer of network structure.

[0209] For each intermediate fusion feature generated in the three-dimensional backbone network, it is input into the next layer of network structure of the feature extraction layer that generates the intermediate fusion feature, wherein the next layer of network structure of the feature extraction layer can be another feature extraction layer or other types of network layers, which is not specifically limited in the embodiment of the present invention.

[0210] Sub-step 2053, obtaining the target extraction features output by the three-dimensional backbone network.

[0211] Through the above process, the information of the two-dimensional features to be fused can be fused during the process of the three-dimensional backbone network extracting the three-dimensional features of the target space, thereby improving the information richness and accuracy of the target extraction features output by the three-dimensional backbone network.

[0212] It should be noted that in the target detection network, the trained image feature encoding network can be directly used. During the training process of the target detection network, the image feature encoding network does not need to be trained. The parameters of the image feature encoding network can be fixed, and only other parts of the target detection network need to be trained.

[0213] Optionally, in the process of training the target detection network, the sample data pairs consisting of the sample two-dimensional image and the sample three-dimensional data can also be processed by the data enhancement module, and more data pairs can be generated based on the sample data pairs, thereby increasing the sample size of the training target detection network and enhancing the training effect of the target detection network. Among them, the above-mentioned data enhancement module may include but is not limited to the cut&paste data enhancement module, etc. The cut&paste data enhancement module can be a semantic-based cut&paste data enhancement module or an ordinary cut&paste data enhancement module, which is not specifically limited in the embodiment of the present invention. It should be noted that since the corresponding sample two-dimensional image and sample three-dimensional data need to be generated, the above-mentioned data enhancement module for training the target detection network can be a semantic-based cut&paste multimodal data enhancement module.

[0214] Step 206: input the target extraction feature into the first pooling layer of the first region of interest pooling subnetwork to obtain the first pooling result output by the first pooling layer.

[0215] In the embodiment of the present invention, the target extraction features can be processed by the region of interest pooling network to obtain a three-dimensional target detection result. During the target extraction feature processing, the two-dimensional features to be fused can be further fused to determine a more accurate three-dimensional target detection result.

[0216] Specifically, the region of interest pooling network includes a first region of interest pooling subnetwork (Region of interest pooling, RoI pooling), and the pooling result includes a first pooling result. First, the three-dimensional feature can be extracted to the first region of interest pooling subnetwork, and in the first region of interest processing process, the first pooling result is output by the first pooling layer of the first region of interest pooling subnetwork.

[0217] like Figure 6 As shown, the region of interest pooling network may include a first region of interest pooling subnetwork 67, and the first region of interest pooling subnetwork 67 includes a first pooling layer and a first pooling result 69 output by the first pooling layer.

[0218] It should be noted that in order to further improve the information richness of the first pooling result, the intermediate fusion features and target extraction features output by at least one intermediate layer in the three-dimensional backbone network can also be used together as the input of the region of interest pooling network.

[0219] Step 207: perform feature fusion on the first pooling result and the two-dimensional features to be fused to obtain a first target pooling result.

[0220] For the first pooling result generated during the processing of the first region of interest pooling subnetwork, the first pooling result and the two-dimensional features to be fused can be fused to obtain a first target pooling result, so that the first target pooling result can contain information of the first pooling result and information of the two-dimensional features to be fused at the same time.

[0221] like Figure 6 As shown, the first pooling result 69 can be fused with the two-dimensional features to be fused 63 to obtain a first target pooling result.

[0222] Step 208: input the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer.

[0223] After obtaining the first target pooling result, the first target pooling result can be input into the first detection layer of the first region of interest pooling subnetwork, so that the first detection layer performs target detection on the first target pooling result to obtain a three-dimensional target detection result.

[0224] Optionally, in order to further improve the accuracy of three-dimensional target detection results, in an embodiment of the present invention, a region of interest pooling network composed of multiple region of interest pooling sub-networks cascaded together can be used to extract target features for target detection. For details, please refer to the following sub-steps 2091 to 2094.

[0225] Optionally, step 208 may include:

[0226] Sub-step 2081: input the first target pooling result into the first detection layer of the first region of interest pooling sub-network to obtain the intermediate target detection result output by the first detection layer.

[0227] Sub-step 2082: input the intermediate target detection result into the second pooling layer of the second region of interest pooling sub-network to obtain the second pooling result output by the second pooling layer.

[0228] In the embodiment of the present invention, the region of interest pooling network may further include a second region of interest pooling sub-network, and the pooling result includes a second pooling result. The first region of interest pooling sub-network and the second region of interest pooling sub-network are connected in series.

[0229] Reference Figure 7 , Figure 7 FIG. 4 shows another target detection network schematic diagram provided by an embodiment of the present invention. Figure 7 As shown, the region of interest pooling network 71 may include a first region of interest pooling subnetwork 72 and a second region of interest pooling subnetwork 73 arranged in series, the first region of interest pooling subnetwork 72. The structure of the second region of interest pooling subnetwork 73 may be the same as that of the first region of interest pooling subnetwork 72, and the second region of interest pooling subnetwork 73 may include a second pooling layer and a second detection layer. The intermediate target detection result generated by the first region of interest pooling subnetwork 72 may be further input into the second pooling layer of the second region of interest pooling subnetwork 73 to obtain a second pooling result 74 output by the second pooling layer.

[0230] Sub-step 2083, fusing the second pooling result with the two-dimensional features to be fused to obtain a second target pooling result.

[0231] like Figure 7 As shown, in an embodiment of the present invention, after obtaining the second pooling result, the second pooling result 74 can be fused with the two-dimensional features to be fused to obtain a second target pooling result.

[0232] Sub-step 2084, inputting the second target pooling result into the second detection layer of the second region of interest pooling sub-network to obtain the three-dimensional target detection result output by the second detection layer.

[0233] like Figure 7 As shown, in an embodiment of the present invention, the second target pooling result can be input into the second detection layer of the second region of interest pooling subnetwork 73 to obtain the three-dimensional target detection result output by the second detection layer.

[0234] It should be noted that the region of interest pooling network may further include a third region of interest pooling sub-network, a fourth region of interest pooling sub-network, etc. The embodiment of the present invention does not specifically limit the number of region of interest pooling sub-networks included in the region of interest pooling network.

[0235] In summary, the embodiment of the present invention provides another target detection method, including: extracting features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features; extracting features from a two-dimensional image to obtain two-dimensional features to be fused; wherein the two-dimensional image and the point cloud data are captured in the same time and space; extracting features from the spatial three-dimensional features based on a three-dimensional backbone network, and fusing the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain target extraction features output by the three-dimensional backbone network; performing target detection on the target extraction features based on a region of interest pooling network, and fusing the pooling results generated in the target detection process with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network. In the embodiment of the present invention, in the process of performing three-dimensional target detection based on point cloud data, the two-dimensional features to be fused corresponding to the two-dimensional image can be fused, so that in the process of performing three-dimensional target detection, some information unique to the two-dimensional image (such as texture information, etc.) can be considered, and the accuracy of performing three-dimensional target detection can be improved through the complementarity of point cloud information and two-dimensional image, thereby obtaining a high-precision three-dimensional detection result.

[0236] Based on the above embodiments, an embodiment of the present invention further provides a target detection device.

[0237] refer to Figure 8 , Figure 8 A structural block diagram of a target detection device provided by an embodiment of the present invention is shown:

[0238] The spatial three-dimensional feature module 801 is used to extract features from the voxel space determined based on the point cloud data to obtain spatial three-dimensional features;

[0239] The feature module 802 to be fused is used to extract features from the two-dimensional image to obtain two-dimensional features to be fused;

[0240] A feature extraction module 803 is used to extract the spatial three-dimensional features and the two-dimensional features to be fused based on a three-dimensional backbone network to obtain target extracted features output by the three-dimensional backbone network;

[0241] The pooling result fusion module 804 is used to perform target detection on the target extraction features based on the region of interest pooling network, and to fuse the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network.

[0242] Optionally, the feature extraction module includes:

[0243] A spatial three-dimensional feature fusion submodule, used to fuse the spatial three-dimensional feature with the two-dimensional feature to be fused to obtain a target spatial three-dimensional feature;

[0244] The target extraction feature submodule is used to extract the three-dimensional features of the target space based on the three-dimensional backbone network, and fuse the intermediate features generated in the feature extraction process with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network.

[0245] Optionally, the spatial three-dimensional feature fusion submodule includes:

[0246] A mapping matrix submodule, used to obtain a mapping matrix between the point cloud data and the two-dimensional image;

[0247] A first feature element submodule, used to determine, according to the mapping matrix, first feature elements respectively corresponding to each of the spatial three-dimensional feature elements in the two-dimensional feature to be fused;

[0248] The spatial three-dimensional feature element fusion submodule is used to fuse each spatial three-dimensional feature element in the spatial three-dimensional feature with the first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0249] Optionally, the spatial three-dimensional feature element fusion submodule includes:

[0250] The feature element to be fused submodule is used to determine the feature element to be fused corresponding to the spatial three-dimensional feature element based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional feature to be fused;

[0251] The target spatial three-dimensional feature generation submodule is used to fuse each spatial three-dimensional feature element in the spatial three-dimensional feature with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

[0252] Optionally, the target extraction feature submodule includes:

[0253] An intermediate fusion feature submodule, for fusing at least one intermediate feature generated when the three-dimensional backbone network extracts the three-dimensional features of the target space with the two-dimensional features to be fused, to obtain an intermediate fusion feature corresponding to the intermediate feature;

[0254] An intermediate fusion feature input submodule, used to input the intermediate fusion feature into the next layer of network structure;

[0255] The target extraction feature acquisition submodule is used to obtain the target extraction feature output by the three-dimensional backbone network.

[0256] Optionally, the intermediate fusion feature submodule includes:

[0257] An upsampling intermediate feature submodule, used to perform upsampling processing on the intermediate feature to obtain an upsampled intermediate feature with the same feature resolution as the spatial three-dimensional feature;

[0258] A second feature element submodule, for determining, according to a mapping matrix between the point cloud data and the two-dimensional image, second feature elements corresponding to each of the up-sampled intermediate feature elements in the up-sampled intermediate feature and in the two-dimensional feature to be fused;

[0259] A fusion intermediate feature submodule, used for fusing each upsampled intermediate feature element in the upsampled intermediate feature with a second feature element corresponding to the upsampled intermediate feature element to obtain a fused intermediate feature;

[0260] The intermediate fusion feature generation submodule is used to downsample the fused intermediate features to obtain the intermediate fusion features having the same feature resolution as the intermediate features.

[0261] Optionally, the region of interest pooling network includes a first region of interest pooling subnetwork, the pooling result includes a first pooling result, and the pooling result fusion module includes:

[0262] A first pooling result submodule, used for inputting the target extraction feature into the first pooling layer of the first region of interest pooling subnetwork, to obtain the first pooling result output by the first pooling layer;

[0263] A first target pooling result submodule, used for performing feature fusion on the first pooling result and the two-dimensional features to be fused to obtain a first target pooling result;

[0264] The three-dimensional target detection result submodule is used to input the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer.

[0265] Optionally, the region of interest pooling network includes a second region of interest pooling subnetwork, the pooling result includes a second pooling result, and the three-dimensional object detection result submodule includes:

[0266] An intermediate target detection result submodule, used for inputting the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain the intermediate target detection result output by the first detection layer;

[0267] A second pooling result submodule, used for inputting the intermediate target detection result into the second pooling layer of the second region of interest pooling subnetwork, to obtain the second pooling result output by the second pooling layer;

[0268] A second target pooling result acquisition submodule, used to fuse the second pooling result with the two-dimensional features to be fused to obtain a second target pooling result;

[0269] The second target pooling result is input into the second detection layer of the second region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the second detection layer.

[0270] Optionally, the feature modules to be fused include:

[0271] A feature extraction submodule, configured to extract features from the two-dimensional image based on an image feature encoding network, and obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image; wherein the two-dimensional image features include target size features, deviation correction features, and detection frame position features output by the image feature encoding network, and at least one of temporary features generated during feature detection of the two-dimensional image by the image feature encoding network;

[0272] The two-dimensional feature to be fused submodule is used to determine the two-dimensional feature to be fused corresponding to the two-dimensional image based on the two-dimensional image feature and the semantic segmentation mask feature.

[0273] Optionally, the two-dimensional feature submodule to be fused includes:

[0274] An up-sampling two-dimensional image feature submodule, used to perform up-sampling processing on the two-dimensional image feature to obtain an up-sampled two-dimensional image feature having a feature resolution that is the same as the feature resolution of the semantic segmentation mask feature;

[0275] The two-dimensional feature to be fused generating submodule is used to perform feature fusion processing on the up-sampled two-dimensional image feature and the semantic segmentation mask feature to obtain the two-dimensional feature to be fused.

[0276] Optionally, the device further comprises:

[0277] An initial feature coding network module, used for inputting a sample two-dimensional image into the initial feature coding network, and obtaining sample two-dimensional image features and sample semantic segmentation mask features output by the initial feature coding network;

[0278] A reference semantic segmentation mask feature module is used to input the sample two-dimensional image into a semantic segmentation model to obtain a reference semantic segmentation mask feature output by the semantic segmentation model;

[0279] The image feature encoding network training module is used to train the initial feature encoding network based on the reference semantic segmentation mask features and the label two-dimensional image features corresponding to the sample two-dimensional image features to obtain the image feature encoding network.

[0280] In summary, an embodiment of the present invention provides a target detection device, including: a spatial three-dimensional feature module, which is used to extract features from a voxel space determined based on point cloud data to obtain spatial three-dimensional features; a feature module to be fused, which is used to extract features from a two-dimensional image to obtain two-dimensional features to be fused; wherein the two-dimensional image and the point cloud data are taken at the same time and space; a feature extraction module, which is used to extract features from spatial three-dimensional features based on a three-dimensional backbone network, and fuse the intermediate features generated in the feature extraction process with the two-dimensional features to be fused, to obtain target extraction features output by the three-dimensional backbone network; a pooling result fusion module, which is used to perform target detection on the target extraction features based on a region of interest pooling network, and fuse the pooling results generated in the target detection process with the two-dimensional features to be fused, to obtain a three-dimensional target detection result output by the region of interest pooling network. In an embodiment of the present invention, in the process of performing three-dimensional target detection based on point cloud data, the two-dimensional features to be fused corresponding to the two-dimensional image can be fused, so that in the process of performing three-dimensional target detection, some information unique to the two-dimensional image (such as texture information, etc.) can be considered, and the accuracy of performing three-dimensional target detection can be improved through the complementarity of point cloud information and two-dimensional image, thereby obtaining a high-precision three-dimensional detection result.

[0281] An embodiment of the present invention further provides a vehicle controller, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the above-mentioned target detection method.

[0282] An embodiment of the present invention further provides a readable storage medium. When the instructions in the readable storage medium are executed by a processor of a vehicle controller, the vehicle controller can execute the above-mentioned target detection method.

[0283] An embodiment of the present invention further provides a vehicle, comprising the above-mentioned vehicle controller.

[0284] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned device embodiments and will not be repeated here.

[0285] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0286] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: The method comprises: Extract features from the voxel space determined based on the point cloud data to obtain three-dimensional features of the space; Extract features from the two-dimensional image to obtain two-dimensional features to be fused; Performing feature extraction on the spatial three-dimensional features and the two-dimensional features to be fused based on the three-dimensional backbone network to obtain target extraction features output by the three-dimensional backbone network; Target detection is performed on the target extraction features based on the region of interest pooling network, and the pooling results generated during the target detection process are fused with the two-dimensional features to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network.

2. The target detection method according to claim 1, characterized in that: The extracting the spatial three-dimensional features and the two-dimensional features to be fused based on the three-dimensional backbone network to obtain the target extracted features output by the three-dimensional backbone network includes: Fusing the spatial three-dimensional features with the two-dimensional features to be fused to obtain target spatial three-dimensional features; The three-dimensional features of the target space are extracted based on the three-dimensional backbone network, and the intermediate features generated in the feature extraction process are fused with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network.

3. The target detection method according to claim 2, characterized in that: The fusing the spatial three-dimensional features with the two-dimensional features to be fused to obtain the target spatial three-dimensional features includes: Acquire a mapping matrix between the point cloud data and the two-dimensional image; Determine, according to the mapping matrix, first feature elements in the two-dimensional features to be fused corresponding to the respective three-dimensional feature elements in the three-dimensional features; Each spatial three-dimensional feature element in the spatial three-dimensional feature is fused with a first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

4. The target detection method according to claim 3, characterized in that: The fusing each spatial three-dimensional feature element in the spatial three-dimensional feature with the first feature element corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature includes: Determine the feature element to be fused corresponding to the spatial three-dimensional feature element based on the first feature element corresponding to the spatial three-dimensional feature element and the adjacent feature elements of the first feature element in the two-dimensional feature to be fused; Each spatial three-dimensional feature element in the spatial three-dimensional feature is fused with the feature element to be fused corresponding to the spatial three-dimensional feature element to obtain the target spatial three-dimensional feature.

5. The target detection method according to claim 2, characterized in that: The three-dimensional features of the target space are extracted based on the three-dimensional backbone network, and the intermediate features generated in the feature extraction process are fused with the two-dimensional features to be fused to obtain the target extraction features output by the three-dimensional backbone network, including: For at least one intermediate feature generated when the three-dimensional backbone network extracts the three-dimensional features of the target space, fusing the intermediate feature with the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature; Input the intermediate fusion features into the next layer of network structure; The target extraction features output by the three-dimensional backbone network are obtained.

6. The target detection method according to claim 5, characterized in that: The fusing the intermediate feature with the two-dimensional feature to be fused to obtain an intermediate fused feature corresponding to the intermediate feature includes: Performing upsampling processing on the intermediate features to obtain upsampled intermediate features having a feature resolution that is the same as a feature resolution of the spatial three-dimensional features; Determine, according to a mapping matrix between the point cloud data and the two-dimensional image, second feature elements corresponding to each of the up-sampled intermediate feature elements in the up-sampled intermediate feature and in the two-dimensional feature to be fused; fusing each upsampled intermediate feature element in the upsampled intermediate feature with a second feature element corresponding to the upsampled intermediate feature element to obtain a fused intermediate feature; Down-sampling is performed on the fused intermediate features to obtain the intermediate fused features having a feature resolution that is the same as the feature resolution of the intermediate features.

7. The target detection method according to claim 1, characterized in that: The region of interest pooling network includes a first region of interest pooling subnetwork, the pooling result includes a first pooling result, the target detection is performed on the target extraction feature based on the region of interest pooling network, and the pooling result generated in the target detection process is fused with the two-dimensional feature to be fused to obtain a three-dimensional target detection result output by the region of interest pooling network, including: Inputting the target extraction feature into the first pooling layer of the first region of interest pooling subnetwork to obtain the first pooling result output by the first pooling layer; Performing feature fusion on the first pooling result and the two-dimensional features to be fused to obtain a first target pooling result; The first target pooling result is input into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer.

8. The target detection method according to claim 7, characterized in that: The region of interest pooling network includes a second region of interest pooling subnetwork, the pooling result includes a second pooling result, and the first target pooling result is input into the first detection layer of the first region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the first detection layer, including: Inputting the first target pooling result into the first detection layer of the first region of interest pooling subnetwork to obtain an intermediate target detection result output by the first detection layer; Inputting the intermediate target detection result into the second pooling layer of the second region of interest pooling subnetwork to obtain the second pooling result output by the second pooling layer; Fusing the second pooling result with the two-dimensional features to be fused to obtain a second target pooling result; The second target pooling result is input into the second detection layer of the second region of interest pooling subnetwork to obtain the three-dimensional target detection result output by the second detection layer.

9. The target detection method according to claim 1, characterized in that: The step of extracting features from the two-dimensional image to obtain two-dimensional features to be fused includes: Extracting features from the two-dimensional image based on an image feature encoding network to obtain two-dimensional image features and semantic segmentation mask features corresponding to the two-dimensional image; wherein the two-dimensional image features include target size features, deviation correction features, and detection frame position features output by the image feature encoding network, and at least one of temporary features generated during feature detection of the two-dimensional image by the image feature encoding network; Based on the two-dimensional image features and the semantic segmentation mask features, two-dimensional features to be fused corresponding to the two-dimensional image are determined.

10. The target detection method according to claim 9, characterized in that: The determining, based on the two-dimensional image feature and the semantic segmentation mask feature, the two-dimensional features to be fused corresponding to the two-dimensional image includes: Performing upsampling processing on the two-dimensional image features to obtain upsampled two-dimensional image features having a feature resolution that is the same as a feature resolution of the semantic segmentation mask features; The upsampled two-dimensional image features and the semantic segmentation mask features are subjected to feature fusion processing to obtain the two-dimensional features to be fused.

11. A target detection device, characterized in that: The device comprises: A spatial three-dimensional feature module is used to extract features from the voxel space determined based on the point cloud data to obtain spatial three-dimensional features; The feature module to be fused is used to extract features from the two-dimensional image to obtain two-dimensional features to be fused; A feature extraction module, used for extracting the spatial three-dimensional features and the two-dimensional features to be fused based on a three-dimensional backbone network, and obtaining target extraction features output by the three-dimensional backbone network; The pooling result fusion module is used to perform target detection on the target extraction features based on the region of interest pooling network, and to fuse the pooling results generated during the target detection process with the two-dimensional features to be fused to obtain the three-dimensional target detection results output by the region of interest pooling network.

12. A vehicle controller, characterized in that: The vehicle controller includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the target detection method according to any one of claims 1 to 10 is implemented.

13. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor of a vehicle controller, the vehicle controller is enabled to execute the target detection method as described in any one of claims 1 to 10.

Images