Target detection method, equipment, device and storage medium

Abstract

The embodiment of the invention relates to a target detection method, equipment and device and a storage medium, and the method comprises the steps: obtaining the image data and point cloud data of a target region, determining the pixel coordinate offset of a feature contained in the image data relative to the feature in the point cloud data based on the point cloud data and the image data, and obtaining the pixel coordinate offset of the feature in the point cloud data; the first bev feature map is determined on the basis of the image data and the pixel coordinate offset, the pixel coordinate offset of the features in the image data is considered when the first bev feature map is determined, and the feature positions in the image data can be corrected through the pixel coordinate offset, so that the correctness of the feature positions in the image data is improved. Therefore, the problem of position deviation during feature fusion caused by time mismatching of the image data and the point cloud data is solved, and after the second bev feature map is obtained based on the point cloud data, target detection is performed based on the first bev feature map and the second bev feature map, so that the accuracy of target detection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of target perception technology, and in particular to a target detection method, device, apparatus, and storage medium. Background Technology

[0002] Related technologies can perceive objects on the road based on data from multiple sensors (such as cameras and radar) installed on both sides of the road, and send the perception results to vehicles to assist driving. However, in practice, sensor data suffers from transmission delays, and there are significant time errors between data from different sensors, especially camera data. Due to the large data volume and the need for encoding and decoding, camera data and radar data have significant time errors. Data with time errors leads to feature mismatch during multi-sensor feature fusion, resulting in low accuracy of multi-sensor fusion perception. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a target detection method, device, apparatus, and storage medium.

[0004] A first aspect of this disclosure provides a target detection method, the method comprising:

[0005] Acquire image data and point cloud data of the target area;

[0006] Based on point cloud data and image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data;

[0007] Based on image data and pixel coordinate offsets, the first bird's-eye view (BEV) feature map is determined;

[0008] Based on point cloud data, determine the second BEV feature map;

[0009] Target detection is performed based on the first and second BEV feature maps.

[0010] In some implementations, determining the pixel coordinate offset of a feature contained in the image data relative to the feature in the point cloud data, based on point cloud data and image data, includes:

[0011] The point cloud data is projected onto the first image coordinate system where the image data is located to obtain the projected data;

[0012] Feature fusion is performed on the projection data and image data to obtain fused data;

[0013] The fused data is processed based on a pre-defined prediction network to obtain the pixel coordinate offsets of the features contained in the image data.

[0014] In some implementations, feature fusion is performed on the projection data and image data to obtain fused data, including:

[0015] The projection data and image data at the same pixel coordinate position are merged, and the merged data is used as the data at the pixel coordinate position.

[0016] In some implementations, determining the first BEV feature map based on image data and pixel coordinate offsets includes:

[0017] Based on image data, a first fusion feature and a ground-free height distribution feature of the image data are determined, wherein the first fusion feature is obtained by fusing the ground-free height distribution feature and the features contained in the image data;

[0018] The position of the first fused feature is corrected based on the pixel coordinate offset to obtain the second fused feature;

[0019] Based on the second fusion feature, the first BEV feature map is determined.

[0020] A second aspect of this disclosure provides a computer device, the computer device comprising:

[0021] Memory and processor;

[0022] Memory is used to store computer programs; processor is used to read computer programs from memory and perform the following operations:

[0023] Acquire image data and point cloud data of the target area;

[0024] Based on point cloud data and image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data;

[0025] The first BEV feature map is determined based on image data and pixel coordinate offsets;

[0026] Based on point cloud data, determine the second BEV feature map.

[0027] Target detection is performed based on the first and second BEV feature maps.

[0028] In some implementations, the processor is used for:

[0029] The point cloud data is projected onto the first image coordinate system where the image data is located to obtain the projected data;

[0030] Feature fusion is performed on the projection data and image data to obtain fused data;

[0031] The fused data is processed based on a pre-defined prediction network to obtain the pixel coordinate offsets of the features contained in the image data.

[0032] In some implementations, the processor is used for:

[0033] The projection data and image data at the same pixel coordinate position are merged, and the merged data is used as the data at the pixel coordinate position.

[0034] In some implementations, the processor is used for:

[0035] Based on image data, a first fusion feature and a ground-free height distribution feature of the image data are determined, wherein the first fusion feature is obtained by fusing the features contained in the image data and the ground-free height distribution feature;

[0036] The position of the first fused feature is corrected based on the pixel coordinate offset to obtain the second fused feature;

[0037] Based on the second fusion feature, the first BEV feature map is determined.

[0038] A third aspect of this disclosure provides a target detection apparatus, comprising:

[0039] The acquisition module is used to acquire image data and point cloud data of the target area;

[0040] The first determining module is used to determine the pixel coordinate offset of the features contained in the image data relative to the features in the point cloud data, based on the point cloud data and the image data.

[0041] The second determining module is used to determine the first bird's-eye view BEV feature map based on image data and pixel coordinate offset;

[0042] The third determination module is used to determine the second BEV feature map based on point cloud data;

[0043] The detection module is used to perform target detection based on the first BEV feature map and the second BEV feature map.

[0044] A fourth aspect of this disclosure provides a processor-readable storage medium storing a program for causing a processor to perform the method described in the first aspect above.

[0045] The technical solution provided in this disclosure has the following advantages compared with related technologies:

[0046] In this embodiment, image data and point cloud data of the target area are acquired. Based on the point cloud data and image data, the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data is determined. Then, a first BEV feature map is determined based on the image data and the pixel coordinate offset. Since the pixel coordinate offset of the feature in the image data is considered when determining the first BEV feature map, the feature position in the image data can be corrected by the pixel coordinate offset. This solves the problem of positional deviation caused by the time mismatch between image data and point cloud data during feature fusion. After obtaining the second BEV feature map based on the point cloud data, target detection is performed based on the first BEV feature map and the second BEV feature map, which can improve the accuracy of target detection. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0048] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a target detection method provided in an embodiment of this disclosure;

[0050] Figure 2 This is a schematic flowchart of a target detection method provided in an embodiment of this disclosure;

[0051] Figure 3 This is a schematic diagram of a method for determining pixel coordinate offset provided in an embodiment of this disclosure;

[0052] Figure 4 This is a schematic diagram of a prediction network provided in an embodiment of this disclosure;

[0053] Figure 5 This is a schematic diagram of a target detection method provided in an embodiment of this disclosure;

[0054] Figure 6 This is a flowchart of a method for determining a BEV feature map provided in an embodiment of this disclosure;

[0055] Figure 7 This is a schematic diagram of a method for BEV feature extraction provided in an embodiment of this disclosure;

[0056] Figure 8This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0057] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0058] In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0060] When performing target detection (such as obstacle detection, but not limited to obstacle detection), related technologies first extract BEV features from the data collected by each type of sensor (such as cameras and radar). Then, they fuse the BEV features from each sensor to obtain a single feature vector. This feature vector is then processed using pattern recognition methods to obtain the target detection result. However, in roadside fusion perception scenarios, these technologies ignore the time delay in sensor data collection, leading to feature mismatch issues during BEV feature fusion and reducing the accuracy of target detection.

[0061] To address the aforementioned problems in related technologies, this disclosure provides a target detection method, for example, Figure 1 This is a schematic diagram of a target detection method provided in an embodiment of this disclosure, as shown below. Figure 1As shown, in some embodiments of this disclosure, the input data can be image data and point cloud data obtained by image acquisition and point cloud acquisition of the same area. In some embodiments, after obtaining the image data and point cloud data, a preset preprocessing operation can be performed on the image data and point cloud data (e.g., image distortion correction, removal of point cloud data with values ​​outside a preset data range, etc., but not limited to the preprocessing operations listed here) to obtain preprocessed image data and point cloud data. Then, based on the preprocessed image data and point cloud data, the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data is determined, and the position of the feature in the image data is corrected based on the pixel coordinate offset. A first BEV feature map is extracted based on the corrected data. Furthermore, by fusing the first BEV feature map with the second BEV feature map extracted from the point cloud data, the fused features are input into the target detection model to obtain the detection result.

[0062] In this embodiment of the disclosure, by using image data and point cloud data, the pixel coordinate offset of a feature contained in the image data relative to the feature in the point cloud data is determined, and the position of the feature in the image data is corrected based on the pixel coordinate offset. This can solve the feature mismatch problem that occurs during BEV feature fusion and improve the accuracy of target detection.

[0063] The solutions provided in this disclosure will be described below with reference to exemplary embodiments.

[0064] Example, Figure 2 This is a flowchart illustrating a target detection method provided in an embodiment of this disclosure. The method can be executed by a computer device, which can be understood as any device with computing and processing capabilities. In this embodiment, for ease of understanding, the computer device can be understood as a server capable of performing target detection (e.g., obstacle detection, but not limited to obstacle detection) based on image data and point cloud data, but is not limited to the server referred to herein. Figure 2 As shown, in some embodiments, the target detection method provided by this disclosure includes steps 201-205:

[0065] Step 201: Obtain image data and point cloud data of the target area.

[0066] The image data and point cloud data mentioned in the embodiments of this disclosure are data collected for the same area (i.e., the target area).

[0067] Image data can be acquired, for example, by cameras (such as security cameras, but not limited to) installed on one or both sides of the road.

[0068] Point cloud data can be acquired, for example, by radar (such as lidar and / or millimeter-wave radar) installed on one or both sides of a road.

[0069] For example, in some implementations, after acquiring the image data and point cloud data of the target area, a data preprocessing step may be included. This involves preprocessing the image data and / or point cloud data based on a preset preprocessing method, such as distortion correction of the image data or deletion of point cloud data with values ​​outside a preset range. Of course, the preprocessing operations listed here are merely illustrative examples and not the only limitations. In practice, the preprocessing operations for image data and / or point cloud data can be limited as needed, and are not necessarily limited to one or a few specific preprocessing operations.

[0070] Step 202: Based on point cloud data and image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data.

[0071] Example, Figure 3 This is a schematic diagram of a method for determining pixel coordinate offset provided in an embodiment of this disclosure. Figure 3 As shown, Figure 3 The image data includes pixel RGB data and image size data. The point cloud data includes the x, y, and z coordinates of each three-dimensional point in the point cloud data, as well as reflectance data. The coordinate transformation parameters include transformation parameters used to project the point cloud data to the first image coordinate system where the image data is located. For example, the extrinsic parameters and intrinsic parameters of the camera coordinate system from the radar coordinate system where the point cloud data is located. The method for projecting the point cloud data to the first image coordinate system based on the preset coordinate transformation parameters can be found in related technologies, and will not be described in detail in the embodiments of this disclosure.

[0072] By projecting the point cloud data onto the first image coordinate system using coordinate transformation parameters, we can obtain the projected data of the point cloud data. This projected data can be understood as a two-dimensional image containing the projected points of the point cloud data. Each pixel in this two-dimensional image corresponds to a three-dimensional point in the point cloud data, and the pixel data includes the x, y, and z coordinates of the corresponding three-dimensional point, as well as its reflectance. Since the projected data and the image data are in the same coordinate system, namely the first image coordinate system, feature fusion processing can be performed on the projected data and the image data to obtain fused data. For example, in one implementation, the x, y, z coordinates and reflectance of the projection point in the projection data and the RGB data of the pixel at the same pixel coordinate position in the image data can be merged to obtain fused data. For example, the data of the three-dimensional point corresponding to the projection point is (x coordinate, y coordinate, z coordinate, reflectance), and the RGB data of the pixel at the same pixel coordinate position in the image data is (R1, G1, B1). Then, the data obtained after merging the data of the projection point and the data of the pixel is (x coordinate, y coordinate, z coordinate, reflectance, R1, G1, B1), and (x coordinate, y coordinate, z coordinate, reflectance, R1, G1, B1) is used as the data of the pixel coordinate position.

[0073] For example, in some implementations, after feature fusion of projection data and image data to obtain fused data, the fused data can be input into a preset prediction network to obtain the pixel coordinate offsets of the features contained in the image data. For example Figure 4 This is a schematic diagram of a prediction network provided in an embodiment of this disclosure, such as... Figure 4 As shown, in some examples, the preset prediction network may include a convolutional network (conv), an activation function (relu), and a pooling module (maxpool). This is achieved by using a preset number of... Figure 1 The number of convolutional networks and pooling modules in the example is only for illustration and not the only limitation. The convolutional network (conv), activation function (re lu), and pooling module (max poo l) extract features from the fused data. Then, the extracted features are input into two convolutional branches, and the horizontal and vertical coordinate offsets of the features are output by the two convolutional branches respectively.

[0074] In other words, in some exemplary descriptions, the method for determining pixel coordinate offsets described above can be described as follows: projecting point cloud data onto a first image coordinate system containing image data to obtain projected data; performing feature fusion on the projected data and image data to obtain fused data; processing the fused data based on a preset prediction network to obtain the pixel coordinate offsets of the features contained in the image data. The scheme for feature fusion of projected data and image data can be described as merging projected data and image data at the same pixel coordinate position, and using the merged data as the data for that pixel coordinate position.

[0075] Step 203: Determine the first BEV feature map based on image data and pixel coordinate offset.

[0076] For example, in one embodiment of this disclosure, the corresponding pixel coordinate offset can be superimposed on the pixels in the image data to obtain image data after feature position correction. Further, the corrected image data is input into a preset first BEV feature extraction model to obtain a first BEV feature map of the image data.

[0077] In another embodiment of this disclosure, the BEV features of the image data can be extracted first based on a preset first BEV feature extraction model, and then the pre-obtained pixel coordinate offset can be superimposed on the BEV features to correct the position of the BEV features, thereby obtaining the first BEV feature map.

[0078] Step 204: Determine the second BEV feature map based on point cloud data.

[0079] For example, in this embodiment of the disclosure, point cloud data can be input into a preset second BEV feature extraction model (such as the Pointpillar network model, but not limited to the Pointpillar network model) to obtain a second BEV feature map.

[0080] Step 205: Perform target detection based on the first BEV feature map and the second BEV feature map.

[0081] Example, Figure 5 This is a schematic diagram of a target detection method provided in an embodiment of this disclosure, as shown below. Figure 5As shown, in some implementations, point cloud data can be processed based on a Pointplotar network model (i.e., a second BEV feature extraction model) to obtain a second BEV feature map of the point cloud data. Further, a preset feature fusion network is used to fuse the first and second BEV feature maps to obtain target fused features. Then, a multi-layer convolutional neural network (CNN) (e.g., a two-layer CNN network, but not limited to two-layer CNN networks) is used to extract features from the target fused features. The extracted features are then input into a preset detection head (e.g., a single-stage object detection (SSD) head) to output target detection results, such as the center coordinates, size, category, and heading angle information of the target object. Of course, Figure 5 This is for illustrative purposes only and not a unique criterion.

[0082] In this embodiment, image data and point cloud data of the target area are acquired. Based on the point cloud data and image data, the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data is determined. Then, a first BEV feature map is determined based on the image data and the pixel coordinate offset. Since the pixel coordinate offset of the feature in the image data is considered when determining the first BEV feature map, the feature position in the image data can be corrected by the pixel coordinate offset. This solves the problem of positional deviation caused by the time mismatch between image data and point cloud data during feature fusion. After obtaining the second BEV feature map based on the point cloud data, target detection is performed based on the first BEV feature map and the second BEV feature map, which can improve the accuracy of target detection.

[0083] Example, Figure 6 This is a flowchart of a method for determining a BEV feature map provided in an embodiment of this disclosure, such as... Figure 6 As shown, in some implementations, the method for determining the first BEV feature map based on image data and pixel coordinate offsets may include steps 601-603.

[0084] Step 601: Based on the image data, determine the first fusion feature and the ground altitude distribution feature of the image data, wherein the first fusion feature is obtained by fusing the ground altitude distribution feature and the features contained in the image data.

[0085] Step 602: Correct the position of the first fused feature based on the pixel coordinate offset to obtain the second fused feature.

[0086] Step 603: Determine the first BEV feature map based on the second fusion feature.

[0087] For example, in related technologies, BEVHeight networks are generally used to extract BEV features from image data. The BEVHeight network employs a height-based 2D-to-3D projection algorithm, using height prediction instead of direct depth prediction, and indirectly converting height to depth through similar triangles to alleviate the optimization process, thus achieving position prediction in 3D space. However, in some embodiments of this disclosure, the BEVHeight network-based BEV feature extraction algorithm can be improved by incorporating pixel coordinate offsets during the 2D-to-3D projection process to correct feature positions, thereby improving the accuracy of BEV feature extraction.

[0088] For example, Figure 7 This is a schematic diagram illustrating a method for BEV feature extraction provided in an embodiment of this disclosure. See also... Figure 7 , Figure 7The encoder in the image data extraction module can be exemplarily composed of a backbone network (e.g., a Swin Transformer network, but not limited to Swin Transformer) and a Feature Pyramid Network (FPN) to extract multi-scale features from the image data. The Height Net, based on the features output by the encoder, extracts contextual features (e.g., relative positional relationships between features) and ground-elevation distribution features from the image data, and fuses these features to obtain a first two-dimensional fused feature. The 2D-to-3D projection module combines the contextual features and ground-elevation distribution features from the image data to project the first fused feature onto the radar's 3D coordinate system to obtain a 3D feature. Specifically, the 2D-to-3D projection module constructs a virtual 3D coordinate system and a reference plane parallel to the image plane at a preset distance. The origin of the virtual 3D coordinate system coincides with the origin of the camera coordinate system, the x-axis and z-axis are parallel to the ground, and the y-axis is perpendicular to the ground. The 2D-to-3D projection module also constructs a reference 3D coordinate system whose coordinate axes are parallel to the corresponding coordinate axes in the radar coordinate system. The origin of this reference 3D coordinate system may or may not coincide with the radar coordinate system. For a point j on the first fused feature, its corresponding pixel coordinates are (u, v), and the coordinates of the corresponding point on the reference plane are (u, v, 1). Assuming that the pixel coordinate offsets corresponding to point j are x11 and y11, the corrected coordinates are (u+x11, v+y11, 1). By multiplying (u+x11, v+y11, 1) by the inverse of the camera intrinsic matrix, we can transform (u+x11, v+y11, 1) into the camera coordinate system, obtaining the corresponding coordinates (x2, y2, z2) in the camera coordinate system. Furthermore, using a pre-set transformation matrix from the camera coordinate system to the virtual 3D coordinate system, we can transform (x2, y2, z2) into the virtual 3D coordinate system, obtaining the corresponding coordinates (x3, y3, z3). Assuming... The virtual 3D coordinate system origin is at a height H above the xoy plane of the reference 3D coordinate system. Then, according to trigonometric theory and a pre-defined transformation matrix from the virtual 3D coordinate system to the reference 3D coordinate system, the coordinates (x3, y3, z3) in the reference 3D coordinate system can be obtained as (x4, y4, z4). The method for projecting points in the virtual coordinate system to the reference 3D coordinate system according to trigonometric theory and a pre-defined transformation matrix from the virtual 3D coordinate system to the reference 3D coordinate system can be found in relevant technologies, and will not be elaborated here.

[0089] Furthermore, if the origin of the reference 3D coordinate system and the origin of the radar coordinate system do not coincide, the process can further include projecting the coordinates (x4, y4, z4) in the reference 3D coordinate system to the radar coordinate system according to a pre-defined transformation matrix between the reference 3D coordinate system and the radar coordinate system, obtaining the coordinates (x5, y5, z5). Then, the volume element pooling module converts (x5, y5, z5) into BEV features, thus obtaining the coordinates of point j in the first BEV feature map. The method for converting the 3D point in the radar coordinate system into BEV features is the same as related technologies and will not be elaborated here.

[0090] For example, if the origin of the reference 3D coordinate system coincides with the origin of the radar coordinate system, then (x4, y4, z4) can be directly converted into a BEV feature to obtain the coordinates of point j in the first BEV feature map. Similarly, each point on the first fused feature can have its coordinates in the first BEV feature map determined using the above method, thus obtaining the first BEV feature map.

[0091] This embodiment corrects the position of the first fused feature of the image data by pixel coordinate offset, and determines the first BEV feature map based on the corrected second fused feature, which can improve the accuracy of the feature position in the first BEV feature map and provide accurate data basis for feature fusion.

[0092] Example, Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Figure 8 As shown, the computer device includes:

[0093] The system includes a memory 81, a transceiver 82, and a processor 83; wherein the memory 81, transceiver 82, and processor 83 are connected to a bus interface.

[0094] Memory 81 is used to store computer programs; transceiver 82 is used to send and receive data under the control of processor 83; processor 83 is used to read the computer program in memory 81 and perform the following operations:

[0095] Acquire image data and point cloud data of the target area;

[0096] Based on point cloud data and image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data;

[0097] The first BEV feature map is determined based on image data and pixel coordinate offsets;

[0098] Based on point cloud data, determine the second BEV feature map.

[0099] Target detection is performed based on the first and second BEV feature maps.

[0100] In some implementations, processor 83 is used for:

[0101] The point cloud data is projected onto the first image coordinate system where the image data is located to obtain the projected data;

[0102] Feature fusion is performed on the projection data and image data to obtain fused data;

[0103] The fused data is processed based on a pre-defined prediction network to obtain the pixel coordinate offsets of the features contained in the image data.

[0104] In some implementations, processor 83 is used for:

[0105] The projection data and image data at the same pixel coordinate position are merged, and the merged data is used as the data at the pixel coordinate position.

[0106] In some implementations, processor 83 is used for:

[0107] Based on image data, a first fusion feature and a ground-free height distribution feature of the image data are determined, wherein the first fusion feature is obtained by fusing the features contained in the image data and the ground-free height distribution feature;

[0108] The position of the first fused feature is corrected based on the pixel coordinate offset to obtain the second fused feature;

[0109] Based on the second fusion feature, the first BEV feature map is determined.

[0110] For example, in one embodiment of this disclosure, a target detection device is provided, comprising:

[0111] The acquisition module is used to acquire image data and point cloud data of the target area;

[0112] The first determining module is used to determine the pixel coordinate offset of the features contained in the image data relative to the features in the point cloud data, based on the point cloud data and the image data.

[0113] The second determining module is used to determine the first bird's-eye view BEV feature map based on image data and pixel coordinate offset;

[0114] The third determination module is used to determine the second BEV feature map based on point cloud data;

[0115] Inspection

[0116] It should be noted that the target detection device provided in this embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0117] It should also be noted that the division of units in the target detection device described in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0118] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this disclosure.

[0119] This disclosure also provides a processor-readable storage medium storing a program for causing a processor to perform the method of any of the above method embodiments.

[0120] In this embodiment of the disclosure, the processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0121] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0122] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A target detection method, characterized in that, The method includes: Acquire image data and point cloud data of the target area; Based on the point cloud data and the image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data; Based on the image data and the pixel coordinate offset, a first bird's-eye view BEV feature map is determined; Based on the point cloud data, a second BEV feature map is determined; Target detection is performed based on the first BEV feature map and the second BEV feature map.

2. The method according to claim 1, characterized in that, The step of determining the pixel coordinate offset of a feature contained in the image data relative to the feature in the point cloud data, based on the point cloud data and the image data, includes: The point cloud data is projected onto the first image coordinate system where the image data is located to obtain the projection data; The projection data and the image data are fused to obtain fused data; The fused data is processed based on a preset prediction network to obtain the pixel coordinate offsets of the features contained in the image data.

3. The method according to claim 2, characterized in that, The step of fusing features from the projection data and the image data to obtain fused data includes: The projection data and image data at the same pixel coordinate position are merged, and the merged data is used as the data at the pixel coordinate position.

4. The method according to claim 1, characterized in that, The step of determining the first bird's-eye view (BEV) feature map based on the image data and the pixel coordinate offset includes: Based on the image data, a first fusion feature and a ground-free height distribution feature of the image data are determined, wherein the first fusion feature is obtained by fusing the ground-free height distribution feature and features contained in the image data; The position of the first fused feature is corrected based on the pixel coordinate offset to obtain the second fused feature; Based on the second fusion feature, the first BEV feature map is determined.

5. A computer device, characterized in that, include: Memory and processor; The memory is used to store computer programs; the processor is used to read the computer programs in the memory and perform the following operations: Acquire image data and point cloud data of the target area; Based on the point cloud data and the image data, determine the pixel coordinate offset of the feature contained in the image data relative to the feature in the point cloud data; Based on the image data and the pixel coordinate offset, a first bird's-eye view BEV feature map is determined; Based on the point cloud data, a second BEV feature map is determined; Target detection is performed based on the first BEV feature map and the second BEV feature map.

6. The computer device according to claim 5, characterized in that, The processor is used for: The point cloud data is projected onto the first image coordinate system where the image data is located to obtain the projection data; The projection data and the image data are fused to obtain fused data; The fused data is processed based on a preset prediction network to obtain the pixel coordinate offsets of the features contained in the image data.

7. The computer device according to claim 6, characterized in that, The processor is used for: The projection data and image data at the same pixel coordinate position are merged, and the merged data is used as the data at the pixel coordinate position.

8. The computer device according to claim 5, characterized in that, The processor is used for: Based on the image data, a first fusion feature and a ground-free height distribution feature of the image data are determined, wherein the first fusion feature is obtained by fusing the features contained in the image data and the ground-free height distribution feature; The position of the first fused feature is corrected based on the pixel coordinate offset to obtain the second fused feature; Based on the second fusion feature, the first BEV feature map is determined.

9. A target detection device, characterized in that, include: The acquisition module is used to acquire image data and point cloud data of the target area; The first determining module is used to determine, based on the point cloud data and the image data, the pixel coordinate offset of a feature contained in the image data relative to the feature in the point cloud data; The second determining module is used to determine the first bird's-eye view BEV feature map based on the image data and the pixel coordinate offset; The third determining module is used to determine the second BEV feature map based on the point cloud data; The detection module is used to perform target detection based on the first BEV feature map and the second BEV feature map.

10. A processor-readable storage medium, characterized in that, The processor-readable storage medium stores a program for causing the processor to perform the method as described in any one of claims 1-4.

Images