A target detection method, apparatus, terminal device, and readable storage medium

Abstract

This application relates to the field of data processing technology and provides a target detection method, apparatus, terminal device, and readable storage medium. Specifically, the target detection method includes: acquiring point cloud data obtained by a radar device scanning a current scene; dividing the scan points in the point cloud data into multiple non-overlapping pillars; sampling scan points within the corresponding pillars based on the distance between each pillar and the radar device, wherein the total number of sampled scan points within each pillar is inversely proportional to the corresponding distance; and performing target detection on the current scene based on the sampled scan points within each pillar. Embodiments of this application can obtain more point cloud information about targets near the radar device while reducing background interference on point cloud information of distant targets, thus improving the accuracy of target detection results.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a target detection method, apparatus, terminal equipment, and readable storage medium. Background Technology

[0002] In target detection tasks based on point cloud data, the traditional approach is to analyze all scan points. This method places high demands on the processing performance of the terminal device and is difficult to meet the real-time requirements of target detection. Compared to images, point cloud data is sparse and disordered. As the target moves further away from the lidar, the number of scan points emitted onto the target decreases sharply. For example, with a 32-line lidar using a 10Hz scanning frequency, only about 12 scan points can be detected on a pedestrian about 25 meters away from the lidar. Randomly sampling scan points at a distance from the lidar device can easily introduce a lot of interference. Therefore, current target detection methods are prone to missed detections and have low accuracy. Summary of the Invention

[0003] This application provides a target detection method, apparatus, terminal device, and readable storage medium, which can solve the problem of low accuracy in current target detection methods.

[0004] A first aspect of this application provides a target detection method, comprising: acquiring point cloud data obtained by a radar device scanning a current scene; dividing the scan points in the point cloud data into multiple non-overlapping pillars; sampling scan points in the corresponding pillars according to the distance between each pillar and the radar device, wherein the total number of scan points sampled in each pillar is inversely proportional to the corresponding distance; and performing target detection on the current scene based on the scan points sampled in each pillar.

[0005] In some embodiments of this application, dividing the scan points in the point cloud data into multiple non-overlapping cylinders includes: mapping the scan points in the point cloud data from a point cloud coordinate system to a two-dimensional coordinate system; dividing the two-dimensional coordinate system into multiple non-overlapping grids, each grid corresponding to a cylinder; and dividing the scan points in the point cloud data into their respective cylinders according to the coordinates of each scan point in the two-dimensional coordinate system.

[0006] In some embodiments of this application, the step of performing target detection on the current scene based on the scan points sampled within each of the plurality of pillars includes: determining initial feature data based on the scan points sampled within each of the plurality of pillars; processing the initial feature data into first feature data of multiple sizes; determining feature data to be fused corresponding to each of the plurality of sizes according to the initial feature data and the first feature data of the plurality of sizes; performing feature fusion on the feature data to be fused corresponding to each of the plurality of sizes to obtain target feature data; and using the target feature data to perform target detection on the current scene.

[0007] In some embodiments of this application, determining the initial feature data based on the scan points sampled within each of the plurality of columns includes: determining the feature value of each column based on the scan points sampled within each of the plurality of columns; and composing the initial feature data from the feature values ​​of each of the plurality of columns.

[0008] In some embodiments of this application, determining the feature data to be fused corresponding to each of the plurality of dimensions based on the initial feature data and the first feature data of the plurality of dimensions includes: performing max pooling on the initial feature data to obtain second feature data that corresponds one-to-one with each of the plurality of dimensions; and fusing the first feature data and the second feature data of the corresponding dimensions to obtain the feature data to be fused corresponding to each of the plurality of dimensions.

[0009] In some embodiments of this application, processing the initial feature data into first feature data of multiple sizes includes: performing a convolution operation on the initial feature data to obtain third feature data for each of the multiple sizes; extracting channel features under a channel attention mechanism and spatial features under a spatial attention mechanism from the third feature data, wherein the channel features are used to characterize the type features of the target in the current scene, and the spatial features are used to characterize the position features of the target in the current scene; and concatenating the channel features and the corresponding spatial features to obtain the first feature data for each of the multiple sizes.

[0010] In some embodiments of this application, the step of using the target feature data to perform target detection on the current scene includes: dividing the target feature data into at least two feature groups; performing an upsampling convolution operation on each feature group to obtain upsampled feature data corresponding to each feature group; concatenating the upsampled feature data corresponding to each feature group to obtain concatenated feature data; and performing target detection on the current scene based on the concatenated feature data.

[0011] A target detection device provided in a second aspect of this application includes: a point cloud data acquisition unit for acquiring point cloud data obtained by a radar device scanning a current scene; a scan point division unit for dividing the scan points in the point cloud data into multiple non-overlapping pillars; a scan point sampling unit for sampling scan points in the corresponding pillars according to the distance between each pillar and the radar device, wherein the total number of scan points sampled in each pillar is inversely proportional to the corresponding distance; and a target detection unit for detecting a target in the current scene based on the scan points sampled in each pillar.

[0012] A third aspect of this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the target detection method described above.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the target detection method described above.

[0014] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the target detection method described in any one of the first aspects.

[0015] In the embodiments of this application, point cloud data obtained by a radar device scanning the current scene is acquired, and the scan points in the point cloud data are divided into multiple non-overlapping pillars. Then, based on the distance between each pillar and the radar device, scan points are sampled in the corresponding pillar. Target detection is then performed on the current scene based on the sampled scan points in each pillar. The total number of sampled scan points in each pillar is inversely proportional to the distance between the corresponding pillar and the radar device. In this way, for targets near the radar device, since there are more scan points emitted onto them, sampling more scan points can obtain more point cloud information about the targets near the radar device, avoiding missed detections due to missing point cloud information. For targets far from the radar device, since there are fewer scan points emitted onto them, sampling fewer scan points can avoid introducing more background information, reducing background interference on target detection and improving the accuracy of target detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of a target detection method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram illustrating the specific implementation process of step S104 provided in the embodiments of this application;

[0019] Figure 3 This is a schematic diagram illustrating the specific implementation process of step S202 provided in the embodiments of this application;

[0020] Figure 4 This is a schematic diagram of the structure of the neural network model provided in the embodiments of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0024] The most commonly used target detection algorithm in the industry is PointPillars. This algorithm can divide a single frame of point cloud data into several pillars in a vertical bird's eye view (BEV) perspective, and then randomly sample the point cloud in each pillar. The number of scan points retained in each pillar is roughly the same, generally 20 to 48, thus forming a 2D feature similar to the BEV plane, and then performing target detection.

[0025] However, compared to images, point cloud data is sparse and disordered. As the target moves further away from the lidar, the number of scan points emitted onto the target decreases dramatically. For example, with a 32-line lidar using a 10Hz scanning frequency, only about 12 scan points can be detected on a pedestrian about 25 meters away from the lidar. When randomly sampling scan points at a distance from the lidar, it is easy to introduce a large amount of point cloud data obtained by scanning the background. For targets close to the lidar, the sampling process easily causes a lot of information loss, resulting in a high false negative rate when the algorithm detects small objects such as pedestrians and bicycles, as well as objects at medium to long distances.

[0026] To address the aforementioned shortcomings, this application can sample scanning points based on the distance between each column and the radar device, ensuring that the total number of scan points sampled within each column is inversely proportional to the corresponding distance. Thus, for targets near the radar device, since a larger number of scan points are emitted onto them, sampling more scan points allows for the acquisition of more point cloud information about the target, avoiding missed detections due to missing point cloud information. Conversely, for targets far from the radar device, since a smaller number of scan points are emitted onto them, sampling fewer scan points avoids introducing more background information, thereby improving the accuracy of target detection.

[0027] To illustrate the technical solution of this application, specific embodiments are described below.

[0028] Figure 1 This illustration shows a schematic diagram of the implementation process of a target detection method provided in an embodiment of this application. This method can be applied to terminal devices and is suitable for situations where improved target detection accuracy is required. The aforementioned terminal device can refer to a computer, mobile phone, robot, unmanned vehicle, roadside sensing device, or other intelligent device.

[0029] Specifically, the above target detection method may include the following steps S101 to S104.

[0030] Step S101: Obtain point cloud data obtained by the radar device scanning the current scene.

[0031] In the embodiments of this application, the current scene can refer to the scene where target detection is required. By installing radar equipment in the current scene, the radar equipment can scan the current scene to obtain point cloud data. In some embodiments, robots or unmanned vehicles can use their own configured radar equipment to scan their area to obtain point cloud data, and the range scanned by the radar equipment is the range of the current scene. In other embodiments, the acquired point cloud data can also be point cloud data obtained by scanning an experimental scene with radar equipment in an experimental environment.

[0032] Specifically, point cloud data can consist of multiple scan points scanned by radar equipment and the point cloud information of these scan points. The point cloud information may include, but is not limited to, the distance between each scan point and the radar equipment, the coordinates of each scan point in the point cloud coordinate system or the radar coordinate system, and the reflectivity corresponding to each scan point.

[0033] It should be noted that this application does not limit the specific model of the radar equipment. The radar equipment can be a rotating, galvanometer-type, or other type of lidar. Furthermore, the scanning frequency and the total number of scanning lines of the radar equipment can be selected according to actual needs.

[0034] Step S102: Divide the scan points in the point cloud data into multiple non-overlapping cylinders.

[0035] In the embodiments of this application, a column can also be called a pillar, which can represent a spatial region within the current scene. By dividing the current scene into multiple non-overlapping columns, the scan points in the point cloud data can be classified and assigned to their respective columns.

[0036] Specifically, after dividing the current scene into multiple non-overlapping cylinders, the coordinate range covered by each cylinder can be obtained. Therefore, based on the coordinates of each scan point in the point cloud coordinate system, it can be determined whether it is located within the coordinate range of a certain cylinder, and then each scan point can be assigned to the cylinder within its coordinate range.

[0037] Considering that the point cloud coordinate system is a three-dimensional coordinate system, the computational load for data processing in a three-dimensional coordinate system is large. Furthermore, when the target detection algorithm serves a robot or autonomous vehicle, the robot and autonomous vehicle focus on the target's position from the BEV's perspective. Therefore, in some embodiments of this application, the scan points in the point cloud data can be mapped from the point cloud coordinate system to a two-dimensional coordinate system. Then, the two-dimensional coordinate system is divided into multiple non-overlapping grids, each grid corresponding to a different cylinder. Based on the coordinates of each scan point in the point cloud data in the two-dimensional coordinate system, the scan points in the point cloud data can be assigned to their respective cylinders.

[0038] The two-dimensional coordinate system can be the coordinate system in the BEV view. Usually, the XoY plane of the point cloud coordinate system is parallel to or coincides with the ground. Therefore, the coordinates (x, y) of each scan point in the point cloud data in the XoY plane of the point cloud coordinate system can be directly extracted to obtain the coordinates of each scan point in the two-dimensional coordinate system.

[0039] In a two-dimensional coordinate system, the coordinates can be divided non-overlappingly from the origin according to the required size of each cylinder, thus rasterizing the two-dimensional coordinate system into multiple grids. For example, the two-dimensional coordinate system from the BEV perspective can be divided into multiple square blocks of equal size as cylinders, with the side length of each square block set to 0.16m to 0.32m. In this case, each grid corresponds to a different coordinate range in the two-dimensional coordinate system. Based on the coordinates (x, y) of each scan point in the two-dimensional coordinate system, it can be determined which grid it belongs to, and thus each scan point in the point cloud data can be assigned to the cylinder corresponding to its assigned grid.

[0040] To improve target detection performance, after acquiring point cloud data, the terminal device can perform preprocessing operations before dividing the preprocessed point cloud data into different cylinders. Specifically, the preprocessing operations may include: obtaining a preset coordinate limit range along each coordinate axis in the point cloud coordinate system, removing scan points whose coordinate values ​​are outside the corresponding coordinate limit range along any coordinate axis, and retaining the remaining scan points. For example, for target detection on a road, the target's height above the ground is generally within 3 meters. Therefore, a coordinate limit range along the Z-axis can be set in the point cloud coordinate system to remove scan points whose height exceeds 3 meters. This reduces background interference to some extent when detecting targets near the radar equipment.

[0041] Step S103: Based on the distance between each of the multiple pillars and the radar device, sample the scanning points within the corresponding pillar.

[0042] The total number of scan points sampled within each cylinder is inversely proportional to the corresponding distance. In other words, the greater the distance between the cylinder and the radar equipment, the farther the scan points within the cylinder are from the radar equipment, and the fewer scan points need to be selected during sampling. Conversely, the smaller the distance between the cylinder and the radar equipment, the closer the scan points within the cylinder are to the radar equipment, and the more scan points need to be selected during sampling.

[0043] At this point, for targets near the radar equipment, the radar emits a large number of scan points towards the target, resulting in a denser point cloud. Therefore, sampling more scan points can retain more point cloud information, ensuring that the retained scan points better cover the target and avoiding missed detections of nearby targets due to missing point cloud information. For targets far from the radar equipment, the radar emits fewer scan points towards the target, resulting in a sparser point cloud. Therefore, sampling fewer scan points can avoid introducing more background information, preventing the introduced background information from interfering with the target detection of distant targets.

[0044] Step S104: Target detection is performed on the current scene based on the scan points sampled within each of the multiple pillars.

[0045] In the embodiments of this application, after sampling each cylinder, features of the target in the current scene can be extracted based on the sampled scan points, thereby achieving target detection in the current scene. The extracted features may include, but are not limited to, reflectivity and relative position to a reference object. Based on multi-frame point cloud data, associated features of different frames of point cloud data can also be obtained, such as the target's motion features. Thus, target detection within the current scene can be achieved, yielding detection results such as the target's type and location. The specific target detection algorithm can be implemented using a one-stage or two-stage algorithm.

[0046] In the embodiments of this application, point cloud data obtained by a radar device scanning the current scene is acquired, and the scan points in the point cloud data are divided into multiple non-overlapping pillars. Then, based on the distance between each pillar and the radar device, scan points are sampled in the corresponding pillar. Target detection is then performed on the current scene based on the sampled scan points in each pillar. The total number of sampled scan points in each pillar is inversely proportional to the distance between the corresponding pillar and the radar device. In this way, for targets near the radar device, since there are more scan points emitted onto them, sampling more scan points can obtain more point cloud information about the targets near the radar device, avoiding missed detections due to missing point cloud information. For targets far from the radar device, since there are fewer scan points emitted onto them, sampling fewer scan points can avoid introducing more background information, reducing background interference on target detection and improving the accuracy of target detection.

[0047] Specifically, such as Figure 2 As shown, step S104 may specifically include steps S201 to S205.

[0048] Step S201: Determine initial feature data based on the scan points sampled in each of the multiple columns.

[0049] Specifically, the terminal device can determine the feature value of each cylinder based on the scan points sampled within each cylinder, and the initial feature data is composed of the feature values ​​of each cylinder. For example, based on the point cloud information of each scan point within the same cylinder, the scan point with the most significant point cloud information in the cylinder can be used as the feature value of that cylinder; alternatively, the average value of the point cloud information of all scan points within the same cylinder can be used as the feature value of the corresponding cylinder. By combining the feature values ​​of each cylinder according to the position of the cylinder in the two-dimensional coordinate system, initial feature data representing the current scene in the two-dimensional coordinate system can be obtained. At this point, the initial feature data can represent 2D features in the BEV plane.

[0050] Specifically, scan points in point cloud data can be represented in multiple dimensions, such as the coordinates (x, y, z) of the scan point in the point cloud coordinate system, reflectivity, and the deviation (x, y, z) between the scan point and the center of the cylinder. c y c , z c The deviation (x) between the coordinates of the scan point in the aforementioned two-dimensional coordinate system and the coordinates of the center of the grid cell in which it is located. p y p , z p Then, a preset quantity threshold can be obtained. If the number of scanned points within a cylinder is less than or equal to the quantity threshold, the cylinder is considered an empty cylinder, which can have a value of 0. If the number of scanned points within a cylinder is greater than the quantity threshold, the cylinder can have the quantity value of the scanned points. Assuming the dimension of the scanned points in the point cloud data is D, and the number of scanned points within a cylinder is M, features of scale (D, M, P) are generated, where P = H × W is used to represent a grid (i.e., a cylinder) in a two-dimensional coordinate system. Furthermore, the features of scale (D, M, P) can be converted into a two-dimensional feature image by performing max pooling operations or other methods in the dimension.

[0051] Step S202: Process the initial feature data into first feature data of multiple sizes.

[0052] In the embodiments of this application, the terminal device can process the initial feature data into first feature data of multiple sizes through convolution processing, and then extract the features in the current scene at different size dimensions, which helps to combine feature information at different resolutions for target detection.

[0053] It should be understood that the number of dimensions mentioned above can be selected according to the actual situation. For example, the initial feature data can be processed into first feature data of three different sizes: 1 / 2, 1 / 4, and 1 / 8 of the initial feature data. Correspondingly, the first feature information of different sizes can be obtained by performing convolution processing on the initial feature data with different convolution kernels.

[0054] Step S203: Based on the initial feature data and the first feature data of multiple dimensions, determine the feature data to be fused for each of the multiple dimensions.

[0055] In the embodiments of this application, each size is treated as a feature branch. The terminal device can determine the feature data to be fused for each size based on the initial feature data and the first feature data of the corresponding size. In this way, the initial column feature (initial feature data) can be passed to subsequent features, preserving the original information and realizing the combination of high semantic features and low semantic features to a certain extent.

[0056] Specifically, the terminal device can perform max pooling on the initial feature data to downsample the initial feature data and obtain second feature data corresponding one-to-one with each of the multiple dimensions. Then, the first feature data and the second feature data corresponding to the dimensions are fused to obtain the feature data to be fused for each of the multiple dimensions.

[0057] For example, after obtaining the initial feature data P1, the initial feature data P1 can be convolved into first feature data B1 at size A1, first feature data B2 at size A2, and first feature data B3 at size A3, where A1, A2, and A3 are different. Simultaneously, the initial feature data P1 is max-pooled into second feature data C1 at size A1, second feature data C2 at size A2, and second feature data C3 at size A3. Then, the first feature data B1 and the second feature data C1 are fused to obtain the feature data to be fused at size A1; the first feature data B2 and the second feature data C2 are fused to obtain the feature data to be fused at size A2; and the first feature data B3 and the second feature data C3 are fused to obtain the feature data to be fused at size A3.

[0058] Step S204: Perform feature fusion on the feature data to be fused for each of the multiple dimensions to obtain the target feature data.

[0059] Step S205: Target detection is performed on the current scene using target feature data.

[0060] Using the aforementioned example, feature fusion of the feature data to be fused (D1 at size A1, D2 at size A2, and D3 at size A3) yields the target feature data. The specific feature fusion method can be concat, add, or other operations; this application does not impose any restrictions on this. Based on the fused target feature data, feature information in the current scene can be extracted, thereby achieving target detection.

[0061] It should be understood that since the target feature data is obtained by fusing data of different sizes, it covers both high-resolution and low-resolution feature information, which can better extract detailed features in the current scene, help reduce the false negative rate of small objects and improve the accuracy of target detection.

[0062] In related technologies, most target detection algorithms based on point cloud data do not use attention mechanisms, or only use channel attention mechanisms to enhance the effect on the target type, ignoring the contribution of spatial channels to the extraction of point cloud information location. In the embodiments of this application, both channel attention mechanisms and spatial attention mechanisms are incorporated when processing feature data.

[0063] Specifically, such as Figure 3 As shown, step S202 can specifically include steps S301 to S303.

[0064] Step S301: Perform a convolution operation on the initial feature data to obtain the third feature data for each of the multiple dimensions.

[0065] The specific process of the convolution operation can be found in the description of step S202 above, and will not be repeated here.

[0066] Step S302: Extract the channel features of the third feature data under the channel attention mechanism and the spatial features under the spatial attention mechanism.

[0067] Among them, channel features can be used to characterize the type features of the target in the current scene, and spatial features can be used to characterize the location features of the target in the current scene.

[0068] Specifically, under each feature branch, channel attention can be extracted first to obtain the channel features of the third feature data under the channel attention mechanism, and then spatial attention can be extracted to obtain the spatial features of the third feature data under the spatial attention mechanism. Alternatively, spatial attention can be extracted first, followed by channel attention.

[0069] The extraction of channel attention specifically involves: extracting high-level features from the third feature data through global average pooling and max pooling, respectively; then processing these high-level features using a fully connected layer and corresponding activation functions; and finally, weighting the processed features onto the third feature data using multiplication to obtain the channel features. Taking a third feature data with an input feature size of 64×512×512 as an example, global average pooling and max pooling can extract 64×1×1 channel features, where 64 represents 64 feature channels. Therefore, the channel attention mechanism can select which channel contributes more to the third feature data. In object detection, the classification result of the target type is often a result that combines the final probability values ​​of each channel. Therefore, channel features focus more on the type characteristics of the target in the current scene.

[0070] Spatial attention extraction also utilizes global average pooling and max pooling. The difference is that this operation is performed along the channel dimension; that is, all input channels are pooled into two real numbers, and the third feature data of shape h×w×c is processed to obtain two h×w×1 feature maps. Then, a convolution kernel is used to form a new h×w×1 spatial feature. Again, using a third feature data with an input feature of 64×512×512 as an example, the final spatial feature can be 1×512×512. Therefore, the spatial attention mechanism can select which pixel contributes more, thus focusing more on the positional features of the target in the current scene.

[0071] Step S303: The channel features and the corresponding spatial features are concatenated to obtain the first feature data of each of the multiple dimensions.

[0072] In the embodiments of this application, under the same feature branch, the channel features and the corresponding spatial features are concatenated. That is, the channel features and spatial features obtained from the same third feature data are concatenated to obtain the first feature data of the size corresponding to the third feature data. By concatenating under each feature branch, the first feature data of each size in multiple sizes can be obtained. It should be understood that the obtained first feature data carries the feature information of the channel features and the feature information of the spatial features. It can be combined with position features and type features to help determine the position of the target, whether the target belongs to the background, and which category the object belongs to in the foreground. This improves the detection rate of small targets (such as pedestrians and bicycles) and makes the final output detection box position (i.e., the position of the target) more accurate.

[0073] In other embodiments of this application, the same processing can be applied to the feature data to be fused described in step S203. Specifically, based on the initial feature data and the first feature data of multiple dimensions, the fourth feature data corresponding to each of the multiple dimensions can be determined, as described in step S203. For the fourth feature data, the channel features under the channel attention mechanism and the spatial features under the spatial attention mechanism can be extracted, and then the channel features and the corresponding spatial features are concatenated to obtain the feature data to be fused for each of the multiple dimensions.

[0074] To ensure timely target detection, the above processing involves multiple downsampling operations, leading to a decrease in the resolution of the feature data and poor granularity of target detection, thus reducing the accuracy of detecting small targets. To preserve high resolution, this application can also perform upsampling on the target feature data. Simultaneously, to ensure efficiency in target detection, grouped convolution can be used during upsampling to reduce computational load. Specifically, the terminal device can divide the target feature data into at least two feature groups, and then perform upsampling convolution on each feature group to obtain upsampled feature data for each group. Next, the upsampled feature data for each feature group is concatenated to obtain concatenated feature data. At this point, the resolution of the concatenated feature data is improved compared to the target feature data. Therefore, using the concatenated feature data to perform target detection in the current scene can improve detection accuracy. Furthermore, the grouped convolution method can reduce the number of parameters required for the convolution operation.

[0075] Please refer to Figure 4 The aforementioned object detection process can be implemented using a neural network model for object detection. This neural network model can include an input layer, a pillar feature processing layer, a backbone layer, an object detection layer (Detection Head), and an output layer. The input layer acquires point cloud data, the pillar feature processing layer processes the point cloud data to obtain initial feature data, the backbone layer processes the initial feature data to obtain object feature data, and the object detection layer performs object detection, which can use a Single-Shot Multibox Detector (SSD). The output layer outputs the object detection results. For detailed processing procedures, please refer to the preceding text. Figures 1 to 4 The description of the above is omitted here. This application improves the training efficiency and capacity of neural network models by fusing high-semantic features and low-semantic features, while reducing the computational cost of neural network models through grouped convolution, thus avoiding overfitting to some extent.

[0076] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.

[0077] like Figure 5 The diagram shown is a structural schematic of a target detection device 500 provided in an embodiment of this application. The target detection device 500 is configured on a terminal device.

[0078] Specifically, the target detection device 500 may include:

[0079] The point cloud data acquisition unit 501 is used to acquire point cloud data obtained by the radar device scanning the current scene;

[0080] The scanning point division unit 502 is used to divide the scanning points in the point cloud data into multiple non-overlapping cylinders;

[0081] The scanning point sampling unit 503 is used to sample scanning points in the corresponding column according to the distance between each column and the radar device, wherein the total number of scanning points sampled in each column is inversely proportional to the corresponding distance.

[0082] The target detection unit 504 is used to perform target detection on the current scene based on the scan points sampled in each of the plurality of pillars.

[0083] In some embodiments of this application, the scanning point division unit 502 described above can be specifically used to: map the scanning points in the point cloud data from the point cloud coordinate system to a two-dimensional coordinate system; divide the two-dimensional coordinate system into multiple non-overlapping grids, each grid corresponding to a cylinder; and divide the scanning points in the point cloud data into their respective cylinders according to the coordinates of each scanning point in the two-dimensional coordinate system.

[0084] In some embodiments of this application, the target detection unit 504 described above can be specifically used to: determine initial feature data based on the scan points sampled within each of the plurality of pillars; process the initial feature data into first feature data of multiple sizes; determine the feature data to be fused corresponding to each of the plurality of sizes according to the initial feature data and the first feature data of the plurality of sizes; perform feature fusion on the feature data to be fused corresponding to each of the plurality of sizes to obtain target feature data; and perform target detection on the current scene using the target feature data.

[0085] In some embodiments of this application, the target detection unit 504 described above can be specifically used to: determine the feature value of each column based on the scan points sampled in each of the plurality of columns; and compose the initial feature data from the feature values ​​of each of the plurality of columns.

[0086] In some embodiments of this application, the target detection unit 504 described above may be specifically used to: perform max pooling on the initial feature data to obtain second feature data that corresponds one-to-one with each of the plurality of sizes; and fuse the first feature data and the second feature data of the corresponding size to obtain feature data to be fused corresponding to each of the plurality of sizes.

[0087] In some embodiments of this application, the target detection unit 504 described above may be specifically used to: perform a convolution operation on the initial feature data processing to obtain third feature data for each of the plurality of dimensions; extract the channel features of the third feature data under the channel attention mechanism and the spatial features under the spatial attention mechanism, wherein the channel features are used to characterize the type features of the target in the current scene, and the spatial features are used to characterize the position features of the target in the current scene; and concatenate the channel features and the corresponding spatial features to obtain the first feature data for each of the plurality of dimensions.

[0088] In some embodiments of this application, the target detection unit 504 described above may be specifically used to: divide the target feature data into at least two feature groups; perform upsampling convolution operations on each feature group to obtain upsampled feature data corresponding to each feature group; concatenate the upsampled feature data corresponding to each feature group to obtain concatenated feature data; and perform target detection on the current scene based on the concatenated feature data.

[0089] It should be noted that, for the sake of convenience and brevity, the specific working process of the target detection device 500 described above can be found in the following reference: Figures 1 to 4 The corresponding process of the method will not be described in detail here.

[0090] like Figure 6 The diagram shown is a schematic of a terminal device provided in an embodiment of this application. The terminal device 6 may include: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a target detection program. When the processor 60 executes the computer program 62, it implements the steps in the various target detection method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The unit shown includes a point cloud data acquisition unit 501, a scan point division unit 502, a scan point sampling unit 503, and a target detection unit 504.

[0091] The computer program can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.

[0092] For example, the computer program can be divided into: a point cloud data acquisition unit, a scan point division unit, a scan point sampling unit, and a target detection unit.

[0093] The specific functions of each unit are as follows: a point cloud data acquisition unit, used to acquire point cloud data obtained by the radar device scanning the current scene; a scan point division unit, used to divide the scan points in the point cloud data into multiple non-overlapping pillars; a scan point sampling unit, used to sample scan points in the corresponding pillars according to the distance between each pillar and the radar device, wherein the total number of scan points sampled in each pillar is inversely proportional to the corresponding distance; and a target detection unit, used to perform target detection in the current scene based on the scan points sampled in each pillar.

[0094] The terminal device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0095] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0096] The memory 61 can be an internal storage unit of the terminal device, such as a hard drive or RAM. The memory 61 can also be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units. The memory 61 is used to store the computer program and other programs and data required by the terminal device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0097] It should be noted that, for the sake of convenience and brevity, the structure of the terminal device described above can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0101] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0105] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target detection method, characterized in that, include: Acquire point cloud data obtained by radar equipment scanning the current scene; The scan points in the point cloud data are divided into multiple non-overlapping cylinders; Based on the distance between each of the plurality of pillars and the radar device, scanning points are sampled in the corresponding pillar, wherein the total number of scanning points sampled in each of the plurality of pillars is inversely proportional to the corresponding distance; Based on the scan points sampled within each of the plurality of pillars, target detection is performed on the current scene; The step of performing target detection on the current scene based on the scan points sampled within each of the plurality of pillars includes: determining initial feature data based on the scan points sampled within each of the plurality of pillars; processing the initial feature data into first feature data of multiple dimensions; determining feature data to be fused corresponding to each of the plurality of dimensions based on the initial feature data and the first feature data of the plurality of dimensions; performing feature fusion on the feature data to be fused corresponding to each of the plurality of dimensions to obtain target feature data; and using the target feature data to perform target detection on the current scene.

2. The target detection method as described in claim 1, characterized in that, The step of dividing the scan points in the point cloud data into multiple non-overlapping cylinders includes: The scan points in the point cloud data are mapped from the point cloud coordinate system to the two-dimensional coordinate system; The two-dimensional coordinate system is divided into multiple non-overlapping grids, each grid corresponding to a cylinder; Based on the coordinates of each scan point in the point cloud data in the two-dimensional coordinate system, the scan points in the point cloud data are divided into their respective cylinders.

3. The target detection method as described in claim 1, characterized in that, The determination of initial feature data based on the scan points sampled within each of the plurality of columns includes: Based on the scan points sampled within each of the plurality of columns, the feature value of each column is determined. The initial feature data is composed of the feature values ​​of each of the plurality of columns.

4. The target detection method as described in claim 1, characterized in that, The step of determining the feature data to be fused for each of the multiple dimensions based on the initial feature data and the first feature data of the multiple dimensions includes: The initial feature data is subjected to max pooling to obtain second feature data that corresponds one-to-one with each of the plurality of dimensions; The first feature data and the second feature data of the corresponding size are fused to obtain the feature data to be fused for each of the plurality of sizes.

5. The target detection method as described in claim 1, characterized in that, The process of processing the initial feature data into first feature data of multiple sizes includes: The initial feature data is processed by convolution to obtain the third feature data for each of the multiple dimensions; Extract the channel features of the third feature data under the channel attention mechanism and the spatial features under the spatial attention mechanism, wherein the channel features are used to characterize the type features of the target in the current scene, and the spatial features are used to characterize the position features of the target in the current scene; The channel features and the corresponding spatial features are concatenated to obtain the first feature data for each of the plurality of dimensions.

6. The target detection method as described in claim 1, characterized in that, The step of using the target feature data to perform target detection in the current scene includes: The target feature data is divided into at least two feature groups; Perform upsampling convolution operations on each of the feature groups to obtain upsampled feature data corresponding to each feature group; The upsampled feature data corresponding to each feature group are concatenated to obtain the concatenated feature data; Based on the stitched feature data, target detection is performed on the current scene.

7. A target detection device, characterized in that, include: The point cloud data acquisition unit is used to acquire point cloud data obtained by radar equipment scanning the current scene; The scanning point division unit is used to divide the scanning points in the point cloud data into multiple non-overlapping cylinders; The scanning point sampling unit is used to sample scanning points in the corresponding column according to the distance between each column and the radar device, wherein the total number of scanning points sampled in each column is inversely proportional to the corresponding distance. The target detection unit is used to perform target detection on the current scene based on the scan points sampled in each of the plurality of pillars; The target detection unit is configured to: determine initial feature data based on the scan points sampled within each of the plurality of pillars; process the initial feature data into first feature data of multiple dimensions; determine the feature data to be fused corresponding to each of the plurality of dimensions based on the initial feature data and the first feature data of the plurality of dimensions; perform feature fusion on the feature data to be fused corresponding to each of the plurality of dimensions to obtain target feature data; and perform target detection on the current scene using the target feature data.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the target detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the target detection method as described in any one of claims 1 to 6.

Images