Target detection method, device, equipment and storage medium

By combining the clustering of point cloud data of vehicle-mounted millimeter-wave radar and target detection network model, the problem of low accuracy in target detection of vehicle-mounted millimeter-wave radar is solved, and the accurate identification of target objects is achieved.

CN113627562BActive Publication Date: 2024-11-26SIWAVE INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110959256.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-20
Publication Date
2024-11-26
Estimated Expiration
2041-08-20

AI Technical Summary

Technical Problem

In target detection, the existing vehicle-mounted millimeter-wave radar has a low accuracy rate of target recognition due to sparse point cloud data.

Method used

By obtaining the point cloud data to be detected, clustering processing is performed to determine the clustered data, and determining the target point in combination with the pre-trained target detection network model, thereby determining the detection box of the target object.

Benefits of technology

The accuracy of target detection is improved. By considering the movement speed of the target object, more accurate clustering results are obtained during the clustering process, and using pre-trained models to predict, accurate detection of the target object is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113627562B_ABST
    Figure CN113627562B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a target detection method, device, equipment and storage medium, the method comprising: obtaining point cloud data to be detected, each point cloud data to be detected includes position coordinates and movement speed; clustering each point cloud data to be detected to determine clustered data; for each group of clustered data, determining the target point based on the clustered data combined with a predetermined target detection network model; determining the target detection frame corresponding to the target object according to each target point. The problem of low accuracy in the target detection process is solved. By clustering the point cloud data to be detected including position coordinates and movement speed, the movement speed of the target object is considered during clustering, and the clustering result is more accurate. Then, each group of clustered data is predicted according to the pre-trained target detection network model to obtain the target point in each group of clustered data, and the target detection frame corresponding to the target object is determined according to each target point, so as to achieve accurate detection of the target object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular to a target detection method, device, equipment and storage medium. Background Art

[0002] With the improvement of living standards, more and more cars have entered family life, providing great convenience for people's lives. In order to provide road risk reminders to vehicle drivers or provide reliable information for autonomous driving path planning, it is becoming increasingly important to accurately identify people, cars, animals and other objects near the vehicle while the car is driving. Common sensors for vehicles are: cameras, lidar and millimeter wave radar. The commonly used vehicle-mounted millimeter wave radar mainly scans the horizontal plane in front, but the point cloud information perceived by the millimeter radar wave is very sparse, resulting in low target recognition accuracy. Summary of the invention

[0003] The present invention provides a target detection method, device, equipment and storage medium to achieve accurate identification of the target.

[0004] In a first aspect, an embodiment of the present invention provides a target detection method, the target detection method comprising:

[0005] Acquire point cloud data to be detected, each of the point cloud data to be detected includes position coordinates and movement speed;

[0006] Clustering each of the point cloud data to be detected to determine clustered data;

[0007] For each group of clustered data, determining a target point based on the clustered data combined with a predetermined target detection network model;

[0008] A target detection frame corresponding to the target object is determined according to each of the target points.

[0009] In a second aspect, an embodiment of the present invention further provides a target detection device, the target detection device comprising:

[0010] An acquisition module is used to acquire point cloud data to be detected, each of which includes position coordinates and movement speed;

[0011] A clustering module, used for clustering the point cloud data to be detected to determine clustered data;

[0012] A target point determination module, for determining a target point for each group of clustered data based on the clustered data combined with a predetermined target detection network model;

[0013] The detection frame determination module is used to determine the target detection frame corresponding to the target object according to each of the target points.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, the device comprising:

[0015] one or more processors;

[0016] a memory for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement a target detection method as described in any one of the embodiments of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a target detection method as described in any one of the embodiments of the present invention.

[0019] The embodiment of the present invention provides a target detection method, device, equipment and storage medium, which obtains point cloud data to be detected, each of which includes position coordinates and movement speed; clusters each of the point cloud data to be detected to determine clustered data; for each group of clustered data, determines the target point based on the clustered data combined with a predetermined target detection network model; and determines the target detection frame corresponding to the target object according to each target point. The problem of low accuracy in the target detection process is solved. By clustering the point cloud data to be detected including position coordinates and movement speed, clustered data of the same type are obtained. Since the movement speed of the target object is considered during clustering, the clustering result is more accurate. Then, each group of clustered data is predicted according to the pre-trained target detection network model to obtain the target point in each group of clustered data, and the target detection frame corresponding to the target object is determined according to each target point, so as to achieve accurate detection of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a target detection method in Embodiment 1 of the present invention;

[0021] Figure 2 is a flow chart of a target detection method in Embodiment 2 of the present invention;

[0022] Figure 3 is a schematic diagram of the structure of a network model to be trained in the second embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of the structure of a target detection device in Embodiment 3 of the present invention;

[0024] Figure 5 It is a structural diagram of a computer device in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings. It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0026] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0027] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0028] Embodiment 1

[0029] Figure 1 A flow chart of a target detection method provided in the first embodiment of the present application is given. The method is applicable to the case where a detection frame with high accuracy is obtained when detecting a target based on point cloud data. The method can be executed by a computer device, which can be composed of two or more physical entities or one physical entity. Generally speaking, the computer device can be a notebook, a desktop computer, a smart tablet, etc.

[0030] It should be noted that the target detection method provided in this embodiment is executed by a computer device, and the computer device can be installed on a vehicle. Accordingly, the application environment of the target detection method implemented in this embodiment can be described as: when the vehicle is driving, point cloud data is collected by a radar installed on the vehicle, and the relevant information of the rectangular box where the target is located in the point cloud data is predicted by processing the point cloud data, thereby obtaining information corresponding to the target. Then the driver can be prompted according to the target information, or the route can be planned according to the target information in automatic driving to avoid vehicle accidents. In the existing implementation scheme, when target detection is performed through point cloud data, detection is performed directly through point cloud data. The objects detected by this target detection method are inaccurate in the recognition of the targets in the point cloud data to be detected due to the sparse data volume of the point cloud data, and the targets in the point cloud data cannot be accurately identified.

[0031] The target detection method provided in this embodiment can accurately identify the target, and by processing the point cloud data to be detected, determine the information corresponding to the target in the point cloud data to be detected, and achieve accurate detection of the target, thereby overcoming the problems existing in the existing technical solutions.

[0032] like Figure 1 As shown, a target detection method provided in this embodiment includes the following steps:

[0033] S110, obtaining point cloud data to be detected, each point cloud data to be detected includes position coordinates and movement speed.

[0034] In this embodiment, the point cloud data to be detected can be specifically understood as point cloud data that needs to be identified by the target, and the point cloud data is collected by the radar. The point cloud data is a massive point set that expresses the spatial distribution of the target and the surface characteristics of the target in the same spatial reference system. The position coordinates can be specifically understood as the coordinate values ​​indicating the location of the point cloud data to be detected, which can be two-dimensional or multi-dimensional. The position coordinates in this application take two-dimensional coordinates as an example. The movement speed is the speed of the point cloud data to be detected during the movement process.

[0035] Specifically, the point cloud data to be detected is collected by radar. In the embodiment of the present application, millimeter wave radar is used as an example. One or more millimeter wave radars are installed on the vehicle to collect environmental information around the vehicle. After the radar collects data, it can be transmitted in real time to a computer device that executes the target detection method, or it can be transmitted to a storage space such as a cloud or a database for storage, and retrieved from the storage space when target detection is required. The point cloud data to be detected collected by the radar includes the position coordinates and movement speed of each point.

[0036] S120: Clustering each of the to-be-detected point cloud data to determine clustered data.

[0037] In this embodiment, the clustered data can be specifically understood as spatial clustered data obtained by dividing the point cloud data to be detected into regions. Each group of clustered data is composed of one or more point cloud data to be detected.

[0038] Specifically, a clustering algorithm is determined in advance, and each point cloud data to be detected is clustered according to the clustering algorithm. By performing a distance search on each point cloud data to be detected, points with similar spatial distances are clustered together to form clustered data.

[0039] S130 . For each group of clustered data, determine the target point based on the clustered data combined with a predetermined target detection network model.

[0040] In the present embodiment, the target detection network model is a neural network model that can perform target recognition. When the target detection network model in the embodiment of the present application recognizes the target, it can predict the probability of each point belonging to a category, and the category can be a car, a person, a background, etc. The target point can be specifically understood as a point that meets the category conditions, and the category of the target point can be set according to the actual target detection needs. For example, the point cloud data to be detected is predicted to be a car point, and the point cloud data to be detected is predicted to be a person point, etc. If the detection of a motor vehicle is taken as an example, the target point is predicted to be a car point (the point refers to the position point represented by the point cloud data to be detected).

[0041] Specifically, the target detection network model is obtained by training a large amount of point cloud data in advance, and the parameters of the model are continuously adjusted according to the loss function during the training process, and finally a target detection network model that meets the requirements is obtained to complete the training. The trained target detection network model can directly input data and obtain the prediction result according to the learning experience. The target detection network model in the present application takes into account the movement speed of the target when predicting, so the prediction result is more accurate. For each group of clustered data, detection is performed separately. A group of clustered data is input into the target detection network model, and the target detection network model predicts the results according to the learning experience to obtain the probability that each point cloud data to be detected belongs to each category, and then the category to which each point cloud data to be detected belongs is determined according to the probability, and then the target point to be obtained in the target detection process is determined according to the category to which each point cloud data to be detected belongs. For example, the target point is a point with a category of motor vehicle, the category of the point cloud data x1 to be detected is motor vehicle, the category of the point cloud data x2 to be detected is human, the category of the point cloud data x3 to be detected is motor vehicle, and the category of the point cloud data x4 to be detected is background, then the point cloud data x1 to be detected and the point cloud data x3 to be detected are target points.

[0042] S140: Determine a target detection frame corresponding to the target object according to each target point.

[0043] In this embodiment, the target object can be specifically understood as an object with detection requirements, and the target object can be a vehicle, a person, a background, etc. The purpose of the target detection method provided in this application is to detect the location of the target object from the point cloud data. The target detection frame can be specifically understood as a polygonal frame used to select the area where the target object is located. The area formed by each target point is the area where the target object is located. A polygonal frame is used to select this area to form a target detection frame.

[0044] The embodiment of the present invention provides a target detection method, which obtains point cloud data to be detected, each of which includes position coordinates and movement speed; clusters each of the point cloud data to be detected to determine clustered data; for each group of clustered data, determines the target point based on the clustered data combined with a predetermined target detection network model; and determines the target detection frame corresponding to the target object according to each of the target points. The problem of low accuracy in the target detection process is solved. By clustering the point cloud data to be detected including position coordinates and movement speed, clustered data of the same type are obtained. Since the movement speed of the target object is considered during clustering, the clustering result is more accurate. Then, each group of clustered data is predicted according to the pre-trained target detection network model to obtain the target point in each group of clustered data, and the target detection frame corresponding to the target object is determined according to each target point, so as to achieve accurate detection of the target object.

[0045] Embodiment 2

[0046] Figure 2 This is a flow chart of a target detection method provided in Embodiment 2 of the present invention. The technical solution of this embodiment is further refined on the basis of the above technical solution, and specifically mainly includes the following steps:

[0047] S210, obtaining point cloud data to be detected, each point cloud data to be detected includes position coordinates and movement speed.

[0048] S220 , normalize and scale each position coordinate and motion speed to obtain a target coordinate and a target speed.

[0049] In this embodiment, the target coordinates can be specifically understood as the coordinate values ​​of the point cloud data to be detected that can be used for clustering after processing, and the target speed can be specifically understood as the movement speed of the point cloud data to be detected that can be used for clustering after processing.

[0050] Specifically, since the values ​​of the distance and movement speed reflected by the position coordinates are relatively large, in order to facilitate clustering, the present application normalizes the position coordinates and the movement speed to unify the measurement dimensions. There are differences in the resolution of the distance and movement speed reflected by the position coordinates, and the normalized distance unit and speed unit interval distribution are uneven, so a proportional factor can be introduced for scaling. The scaling process can scale the position coordinates or the movement speed separately, or both at the same time. If only one of them is selected for scaling, the scaling factor of the other is set to 1. The embodiment of the present application takes the scaling of the movement speed as an example, introduces a proportional factor γ in the speed dimension, and scales the interval of the speed unit. Exemplarily, γ can be selected as 0.5. By normalizing and scaling each position coordinate and movement speed respectively, the target coordinates and target speed are obtained.

[0051] S230 , clustering each target coordinate and target speed to obtain each cluster data.

[0052] Exemplarily, the clustering method may be K-Means (K-means) clustering, mean shift clustering, density-based clustering method (DBSCAN), Gaussian mixture model (GMM) maximum expectation (EM) clustering, etc. This application takes DBSCAN as an example to cluster each target coordinate and target speed to obtain each cluster data.

[0053] DBSCAN performs a distance search on the distance neighborhood (Eps neighborhood) of each point in the data of the clustering space. When it is found that there are more than the set point threshold in the Eps neighborhood of a point p, point p is clustered as a core object; the DBSCAN algorithm iteratively clusters the obtained core objects together. When clustering, some clusters with similar spatial distances will be merged; clustering is completed when no new objects are added to all clusters.

[0054] In the existing clustering methods, clustering is performed only based on distance. However, when detecting targets, taking the scenario of car radar as an example, there is a situation where two cars intersect when they are close to each other, and at a certain moment, there is a slow-walking pedestrian next to a moving car. At this time, clustering is performed only based on distance, and the clustering result is inaccurate. Therefore, the present application adds speed to the clustering algorithm, and uses the target speed to distinguish between two intersecting cars and cars and pedestrians. The introduction of speed can effectively solve some erroneous divisions caused by clustering only with two-dimensional distance coordinates. During clustering, each point (i.e., the point cloud data to be detected) is a three-dimensional vector containing a horizontal coordinate, a vertical coordinate, and a target speed. Among them, the horizontal coordinate and the vertical coordinate constitute the target coordinate. By clustering the target coordinates and target speed of each point cloud data to be detected, each clustered data is obtained, and each group of clustered data contains one or more point cloud data to be detected.

[0055] S240 , for each group of clustered data, input the clustered data into the target detection network model to obtain the model prediction category corresponding to each point cloud data to be detected in the clustered data output by the target detection network model.

[0056] In this embodiment, the model prediction category can be specifically understood as the category of the point cloud data to be detected predicted by the target detection network model, for example, car, person, background, etc.

[0057] Specifically, a group of clustered data is input into the target detection network model, and the target detection network model predicts each point cloud data to be detected in the clustered data based on the pre-learned experience and model parameters, and determines the model prediction category corresponding to each point cloud data to be detected. In the same way, each group of clustered data is input into the target detection network model, and the model prediction category corresponding to each point cloud data to be detected is obtained.

[0058] As an optional embodiment of this embodiment, this optional embodiment further optimizes and includes determining the target detection network model. The steps of determining the target detection network model include A1-A4:

[0059] A1. Obtain training samples, each training sample including a horizontal coordinate, a vertical coordinate, a horizontal velocity component and a vertical velocity component of at least one training point cloud data, and a standard category corresponding to each training point cloud data.

[0060] In this embodiment, the training sample includes a set of data to be learned and the corresponding standard category (i.e., the standard learning result). In this embodiment of the application, the collected point cloud data is used as the point cloud data to be trained, and the point cloud data to be trained and its corresponding standard category are used as a set of data in a training sample. Each training sample in this application contains one or more training point cloud data and its corresponding standard category. The standard category is usually annotated manually or by other means, and the standard category needs to be determined in advance before training.

[0061] Specifically, a large amount of point cloud data is collected in advance, processed by clustering, and n clusters of data are obtained. Each point cloud data in each cluster is used as training point cloud data, aligned and labeled, and standard categories are obtained to complete the data processing before model training. Each cluster of data is used as a training sample. When training the model, the processed training samples can be directly obtained. The horizontal coordinate, vertical coordinate, horizontal velocity component and vertical velocity component of the training point cloud data can be directly collected by the radar.

[0062] A2. Input the corresponding training samples in the current iteration into the current network model to be trained to obtain the training prediction category.

[0063] In this embodiment, the network model to be trained can be specifically understood as an untrained neural network model based on deep learning. The training prediction category can be specifically understood as the result obtained by predicting the category of the training point cloud data during the model training process.

[0064] Specifically, the training samples corresponding to the current iteration are input into the current network model to be trained, and the network model to be trained performs prediction according to the current network parameters to obtain the training prediction category corresponding to each training point cloud data.

[0065] The network model to be trained includes: a first perceptron, a second perceptron, a pooling layer, a fully connected layer, and a third perceptron;

[0066] Among them, the parameters of the first perceptron, the second perceptron and the third perceptron are different.

[0067] In this embodiment, the first perceptron, the second perceptron, and the third perceptron are all perceptrons, and the MLP model is adopted, but the parameters of each perceptron are different. In this embodiment of the application, the first perceptron is MLP (64, 64), the second perceptron is MLP (64, 128, 1024), and the third perceptron is MLP (512, 256, 128, 128, 2) as an example.

[0068] As an optional embodiment of this embodiment, this optional embodiment further inputs the corresponding training samples in the current iteration into the current network model to be trained, and obtains the training prediction category optimized to A21-A27:

[0069] A21. Input the horizontal coordinate, vertical coordinate, horizontal velocity component and vertical velocity component of each training point cloud data into the first perceptron to obtain the first feature data.

[0070] In this embodiment, the first feature data can be specifically understood as data obtained after being processed by the first perceptron, and is used to represent data features of the training point cloud data.

[0071] Specifically, each training point cloud data has a corresponding horizontal coordinate, vertical coordinate, horizontal velocity component and longitudinal velocity component. Each training point cloud data in the training sample under the current iteration is input into the first perceptron. The first perceptron is an MLP model. Since there are connections and influences between the categories of each training point cloud data, that is, the probability that a training point cloud data is a car (or a person, background, etc.) will be affected by other training point cloud data within a certain distance, the first perceptron integrates the coordinates and speeds of each training point cloud data, upgrades the information of each training point cloud data, and obtains the first feature data corresponding to each training point cloud data.

[0072] A22. Input each first feature data into a second perceptron to obtain second feature data.

[0073] In this embodiment, the second feature data can be specifically understood as feature data obtained after being processed by the second perceptron. Each first feature data is input into the second perceptron, and the second feature data is obtained after the second perceptron performs dimensionality-upgrading processing.

[0074] A23. Input each second feature data into a pooling layer to obtain pooled data.

[0075] In this embodiment, the pooled data can be specifically understood as data obtained after pooling by the pooling layer. The pooling layer compresses each second feature data input to reduce the feature map, simplify the network calculation complexity, and simultaneously performs feature compression to extract the main features to obtain pooled data.

[0076] A24. Input the pooled data into the fully connected layer to obtain the region category probability.

[0077] In this embodiment, the regional category probability can be specifically understood as the category to which the region constituted by each training point cloud data in the training sample under the current iteration belongs. Taking FCS (512, 256, k) as an example, the fully connected layer in the embodiment of the present application inputs the pooled data into the fully connected layer, and the fully connected layer extracts k-dimensional global information from the pooled data and predicts the regional category probability, where k is the total number of different categories. The regional category probability of the embodiment of the present application may include only the probability of a car, or it may include the probabilities of multiple categories. For example, k = 3, the probability of a car is p1, the probability of a person is p2, the probability of the background is p3, etc., and the sum of the probability values ​​​​p1+p2+p3=1.

[0078] A25. Perform feature splicing on each first feature data, pooled data and regional category probability to obtain spliced ​​feature data.

[0079] In this embodiment, the spliced ​​feature data can be specifically understood as feature data obtained after splicing the data. For each first feature data, the first feature data, pooled data and regional category probability are feature superimposed to obtain spliced ​​feature data. The number of spliced ​​feature data is the same as the number of first feature data, and the number of first feature data is the same as the number of training point cloud data in the training sample under the current iteration. The regional category probability affects the category probability judgment of each training point cloud data, so the data is spliced ​​and superimposed for probability prediction.

[0080] A26. Input each concatenated feature data into the third perceptron to obtain the predicted type probability.

[0081] In this embodiment, the prediction type probability can be specifically understood as the probability of different predicted types. After each spliced ​​feature data is input into the third sensor, the prediction type probability corresponding to the training point cloud data is obtained, for example, the probability that the training point cloud data is a car and the probability that it is not a car.

[0082] A27. Determine the training prediction category of each training point cloud data according to the probability of each prediction type.

[0083] For each training point cloud data, the corresponding prediction type probabilities are compared, and the prediction type with the larger probability value is determined as the training prediction category of this training point cloud data. For example, if the prediction type probabilities corresponding to a training point cloud data are: the probability of being a car is 80%, and the probability of not being a car is 20%, then the training prediction category of the training point cloud data is car.

[0084] A3. Using at least one given loss function expression, combining each training prediction category and the corresponding standard category, obtain a corresponding loss function, and form a fitting loss function by fusing each loss function.

[0085] In this embodiment, the fitting loss function can be understood as a loss function obtained by fitting multiple loss functions. When back propagating the network model to be trained, a loss function needs to be used. If there are multiple loss functions, multiple loss functions need to be fitted, and then back propagation is performed based on the fitted functions. The loss function can be a GAN loss function, an L1 loss function, a focal loss function, a VGG perceptual loss function, etc.

[0086] For each training point cloud data, a loss function expression is used for calculation according to its corresponding training prediction category and standard category to determine the corresponding loss function, and the loss function corresponding to the remaining training point cloud data is obtained in the same way. One or more loss functions are fitted to obtain a fitted loss function. The fusion processing of each loss function to form a fitted loss function can be performed by setting different weights for each loss function, taking an average value, etc., which is not specifically limited in the embodiments of the present invention.

[0087] A4. Back-propagate the network model to be trained based on the fitting loss function to obtain the network model to be trained for the next iteration until the iterative convergence condition is met to obtain the target detection network model.

[0088] During the training process of the neural network model, the model is continuously updated and adjusted through the back propagation method until the output of the model is consistent with the target. After determining the fitting loss function, the fitting loss function is used to perform back propagation on the network model to be trained to obtain the target detection network model. The embodiment of the present invention does not limit the specific back propagation process and can be set according to the specific situation. After the model training is completed, the target detection network model can be used to realize the category prediction of the point cloud data to be detected.

[0089] For example, Figure 3 A schematic diagram of the structure of a network model to be trained provided in an embodiment of the present invention, the network model to be trained includes: a first perceptron 31, a second perceptron 32, a pooling layer 33, a fully connected layer 34 and a third perceptron 35. It can be known that the structure of the network model to be trained is the same as that of the target detection network model. Taking n training point cloud data in a cluster of data as input, the first perceptron 31 is MLP (64, 64), the second perceptron 32 is MLP (64, 128, 1024), and the third perceptron 35 is MLP (512, 256, 128, 128, 2) as an example, the process of determining the training prediction category is explained.

[0090] The n×4 training point cloud data (where n is the number and 4 is the dimension, similar data below have the same meaning and will not be repeated here, the four dimensions are: horizontal coordinate, vertical coordinate, horizontal velocity component and vertical velocity component) are input into the first perceptron 31 to obtain n×64 first feature data. The n×64 first feature data is input into the second perceptron 32 to obtain the n×1024 second feature data; the n×1024 second feature data is input into the pooling layer 33 to obtain the 1×1024 pooling data; the 1×1024 pooling data is input into the fully connected layer 34 to obtain the k-dimensional regional category probability; the n×64 first feature data, the 1024 pooling data and the k-dimensional regional category probability are feature concatenated to obtain n×(64+1024+k) concatenated feature data; the n×(64+1024+k) concatenated feature data is input into the third perceptron 35 to obtain n×2 predicted type probabilities (for example, the probability of being a car and the probability of not being a car); the n×2 predicted type probabilities are screened to obtain n×1 prediction results, i.e., the training prediction categories of each training point cloud data.

[0091] S250, screening the prediction categories of each model to obtain a target prediction category.

[0092] In this embodiment, the target prediction category can be specifically understood as the category to be obtained when performing target detection based on the point cloud data to be detected. For example, the category of the point cloud data to be detected may be car, pedestrian, background, and the target to be detected in this application is a car, so the target prediction category is car.

[0093] Specifically, according to the detection requirements of the target object, each model prediction category is screened to obtain the target prediction category that meets the conditions. Taking the model prediction category output by the target detection network model as car or non-car as an example, if the target object is a car, the target prediction category is car, that is, the model prediction category of the car category is screened from each model prediction category.

[0094] S260: Determine the point cloud data to be detected corresponding to each target prediction category.

[0095] The target prediction category is obtained by performing category prediction on the point cloud data to be detected, and each target prediction category corresponds to a point cloud data to be detected.

[0096] S270: Determine the coordinate point corresponding to each point cloud data to be detected as the target point.

[0097] S280: Determine the position coordinates of each target point.

[0098] The target point is a point in the point cloud data to be detected, so the target point has corresponding position coordinates.

[0099] S290: Determine the target polygon based on the position coordinates combined with a preset polygon coverage algorithm.

[0100] In this embodiment, the polygonal covering algorithm can be specifically understood as a method for implementing covering points with polygons. For example, a minimum rectangle covering algorithm, such as a convex hull algorithm, a convex hull hull algorithm, etc. The target polygon can be specifically understood as a polygon detection box for selecting the target object, for example, a rectangular detection box. The polygonal covering algorithm is pre-selected, and each position coordinate is selected by the polygonal covering algorithm to generate a target polygon.

[0101] S291. Determine the target polygon as the target detection box corresponding to the target object.

[0102] Directly determine the target polygon as the target detection box to achieve box selection and detection of the target object.

[0103] The embodiment of the present invention provides a target detection method, which solves the problem of low accuracy in the target detection process. By clustering the point cloud data to be detected including position coordinates and movement speed, clustered data of the same type are obtained. Since the movement speed of the target object is considered during clustering, the clustering result is more accurate. One or more training samples are obtained in advance, and the network model to be trained is trained according to the horizontal coordinate, vertical coordinate, horizontal velocity component and vertical velocity component of at least one training point cloud data in the training sample to obtain a target detection network model. Since the target detection network model considers the influence of movement speed when performing category prediction, the prediction result is more accurate. Each group of clustered data is predicted by the target detection network model to obtain the target point in each group of clustered data, and the target detection box corresponding to the target object is determined according to each target point, so as to achieve accurate detection of the target object.

[0104] Embodiment 3

[0105] Figure 4 This is a schematic diagram of the structure of a target detection device provided in Embodiment 3 of the present invention. The device includes: an acquisition module 41, a clustering module 42, a target point determination module 43 and a detection frame determination module 44.

[0106] The acquisition module 41 is used to acquire the point cloud data to be detected, each of which includes position coordinates and movement speed;

[0107] A clustering module 42, used for clustering each of the to-be-detected point cloud data to determine clustered data;

[0108] A target point determination module 43, for determining a target point for each group of clustered data based on the clustered data combined with a predetermined target detection network model;

[0109] The detection frame determination module 44 is used to determine the target detection frame corresponding to the target object according to each of the target points.

[0110] The embodiment of the present invention provides a target detection device, which solves the problem of low accuracy in the target detection process. By clustering the point cloud data to be detected including the position coordinates and the movement speed, clustered data of the same type are obtained. Since the movement speed of the target object is considered during clustering, the clustering result is more accurate. Then, each group of clustered data is predicted according to the pre-trained target detection network model to obtain the target point in each group of clustered data, and the target detection box corresponding to the target object is determined according to each target point, so as to achieve accurate detection of the target object.

[0111] Furthermore, the clustering module 42 includes:

[0112] A target data determination unit, used for normalizing and scaling each of the position coordinates and the motion speed to obtain a target coordinate and a target speed;

[0113] The clustering unit is used to cluster the target coordinates and target speeds to obtain cluster data.

[0114] Furthermore, the device also includes:

[0115] A sample acquisition module, used to acquire training samples, each of which includes a horizontal coordinate, a vertical coordinate, a horizontal velocity component and a vertical velocity component of at least one training point cloud data, and a standard category corresponding to each training point cloud data;

[0116] The category prediction module is used to input the corresponding training samples in the current iteration into the current network model to be trained to obtain the training prediction category;

[0117] A loss function determination module, used to obtain a corresponding loss function by using at least one given loss function expression, combining each of the training prediction categories and the corresponding standard category, and forming a fitting loss function by fusing the loss functions;

[0118] The back propagation module is used to perform back propagation on the network model to be trained based on the fitting loss function to obtain the network model to be trained for the next iteration until the iterative convergence condition is met to obtain the target detection network model.

[0119] Furthermore, the network model to be trained includes: a first perceptron, a second perceptron, a pooling layer, a fully connected layer and a third perceptron;

[0120] The parameters of the first perceptron, the second perceptron and the third perceptron are different.

[0121] Furthermore, the category prediction module includes:

[0122] A first data determination unit, configured to input the abscissa, ordinate, lateral velocity component and longitudinal velocity component of each of the training point cloud data into a first perceptron to obtain first feature data;

[0123] A second data determination unit, configured to input each of the first feature data into the second perceptron to obtain second feature data;

[0124] A pooling data determining unit, used for inputting each of the second feature data into the pooling layer to obtain pooling data;

[0125] A region probability determination unit, used for inputting the pooled data into the fully connected layer to obtain a region category probability;

[0126] A splicing unit, used for performing feature splicing on each of the first feature data, the pooled data and the regional category probability to obtain spliced ​​feature data;

[0127] A type probability determination unit, used for inputting each of the spliced ​​feature data into the third perceptron to obtain a predicted type probability;

[0128] The category prediction unit is used to determine the training prediction category of each of the training point cloud data according to each of the prediction type probabilities.

[0129] Furthermore, the target point determination module 43 includes:

[0130] A model category determination unit, used to input the clustered data into a target detection network model, and obtain a model prediction category corresponding to each point cloud data to be detected in the clustered data output by the target detection network model;

[0131] A screening unit, used to screen the prediction categories of each model to obtain a target prediction category;

[0132] A point cloud data determination unit, used to determine the point cloud data to be detected corresponding to each target prediction category;

[0133] The target point determination unit is used to determine the coordinate point corresponding to each of the point cloud data to be detected as the target point.

[0134] Furthermore, the detection frame determination module 44 includes:

[0135] A coordinate determination unit, used to determine the position coordinates of each of the target points;

[0136] A polygon determination unit, configured to determine a target polygon based on each of the position coordinates in combination with a preset polygon coverage algorithm;

[0137] The detection frame determination unit is used to determine the target polygon as a target detection frame corresponding to the target object.

[0138] The target detection device provided in the embodiment of the present invention can execute the target detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0139] Embodiment 4

[0140] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention is shown in FIG. Figure 5 As shown, the device includes a processor 50, a memory 51, an input device 52 and an output device 53; the number of processors 50 in the device can be one or more. Figure 5A processor 50 is taken as an example; the processor 50, memory 51, input device 52 and output device 53 in the device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0141] The memory 51, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the target detection method in the embodiment of the present invention (for example, the acquisition module 41, the clustering module 42, the target point determination module 43 and the detection frame determination module 44 in the target detection device). The processor 50 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 51, that is, realizing the above-mentioned target detection method.

[0142] The memory 51 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include a memory remotely arranged relative to the processor 50, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The input device 52 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 53 may include a display device such as a display screen.

[0144] Embodiment 5

[0145] Embodiment 5 of the present invention further provides a storage medium including computer executable instructions, wherein the computer executable instructions are used to perform a target detection method when executed by a computer processor, the method comprising:

[0146] Acquire point cloud data to be detected, each of the point cloud data to be detected includes position coordinates and movement speed;

[0147] Clustering each of the point cloud data to be detected to determine clustered data;

[0148] For each group of clustered data, determining a target point based on the clustered data combined with a predetermined target detection network model;

[0149] A target detection frame corresponding to the target object is determined according to each of the target points.

[0150] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided in an embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in the target detection method provided in any embodiment of the present invention.

[0151] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0152] It is worth noting that in the embodiment of the above-mentioned target detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0153] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A target detection method, characterized in that: include: Acquire point cloud data to be detected, each of the point cloud data to be detected includes position coordinates and movement speed; Clustering each of the point cloud data to be detected to determine clustered data; For each group of clustered data, determining a target point based on the clustered data combined with a predetermined target detection network model; Determine a target detection frame corresponding to the target object according to each of the target points; The step of determining the target detection network model includes: Acquire training samples, each of the training samples comprising at least one abscissa, ordinate, lateral velocity component and longitudinal velocity component of a training point cloud data, and a standard category corresponding to each of the training point cloud data; Input the corresponding training samples in the current iteration into the current network model to be trained to obtain the training prediction category; Using at least one given loss function expression, combining each of the training prediction categories and the corresponding standard category, obtaining a corresponding loss function, and forming a fitting loss function by fusing the loss functions; Performing back propagation on the network model to be trained based on the fitting loss function to obtain a network model to be trained for the next iteration, until an iteration convergence condition is met, thereby obtaining a target detection network model; The network model to be trained includes: a first perceptron, a second perceptron, a pooling layer, a fully connected layer and a third perceptron; wherein the parameters of the first perceptron, the second perceptron and the third perceptron are different; The step of inputting the training sample corresponding to the current iteration into the current network model to be trained to obtain the training prediction category includes: Inputting the abscissa, ordinate, lateral velocity component and longitudinal velocity component of each of the training point cloud data into a first perceptron to obtain first feature data; Inputting each of the first feature data into the second perceptron to obtain second feature data; Input each of the second feature data into the pooling layer to obtain pooling data; Inputting the pooled data into the fully connected layer to obtain region category probability; Performing feature splicing on each of the first feature data, pooled data and regional category probability to obtain spliced ​​feature data; Input each of the spliced ​​feature data into the third perceptron to obtain a predicted type probability; The training prediction category of each of the training point cloud data is determined according to each of the prediction type probabilities.

2. The method according to claim 1, characterized in that The step of clustering the point cloud data to be detected to determine clustered data includes: Normalizing and scaling the position coordinates and movement speeds to obtain target coordinates and target speeds; Clustering is performed on the target coordinates and target speeds to obtain cluster data.

3. The method according to claim 1, characterized in that The determining of the target point based on the clustering data combined with a predetermined target detection network model includes: Inputting the clustered data into the target detection network model to obtain the model prediction category corresponding to each point cloud data to be detected in the clustered data output by the target detection network model; Screening the prediction categories of each model to obtain a target prediction category; Determine the point cloud data to be detected corresponding to each target prediction category; The coordinate points corresponding to the point cloud data to be detected are determined as target points.

4. The method according to claim 1, characterized in that: The determining of a target detection frame corresponding to the target object according to each of the target points includes: Determine the position coordinates of each of the target points; Determine the target polygon based on each of the position coordinates in combination with a preset polygon covering algorithm; The target polygon is determined as a target detection frame corresponding to the target object.

5. A target detection device, characterized in that: include: An acquisition module is used to acquire point cloud data to be detected, each of which includes position coordinates and movement speed; A clustering module, used for clustering the point cloud data to be detected to determine clustered data; A target point determination module, for determining a target point for each group of clustered data based on the clustered data combined with a predetermined target detection network model; A detection frame determination module, used to determine a target detection frame corresponding to the target object according to each of the target points; The device also includes: A sample acquisition module, used to acquire training samples, each of which includes a horizontal coordinate, a vertical coordinate, a horizontal velocity component and a vertical velocity component of at least one training point cloud data, and a standard category corresponding to each training point cloud data; The category prediction module is used to input the corresponding training samples in the current iteration into the current network model to be trained to obtain the training prediction category; A loss function determination module, used to obtain a corresponding loss function by using at least one given loss function expression, combining each of the training prediction categories and the corresponding standard category, and forming a fitting loss function by fusing the loss functions; A back propagation module, used for back propagating the network model to be trained based on the fitting loss function to obtain the network model to be trained for the next iteration until the iterative convergence condition is met to obtain the target detection network model; The network model to be trained includes: a first perceptron, a second perceptron, a pooling layer, a fully connected layer and a third perceptron, and the parameters of the first perceptron, the second perceptron and the third perceptron are different; The category prediction module comprises: A first data determination unit, configured to input the abscissa, ordinate, lateral velocity component and longitudinal velocity component of each of the training point cloud data into a first perceptron to obtain first feature data; A second data determination unit, configured to input each of the first feature data into the second perceptron to obtain second feature data; A pooling data determining unit, used for inputting each of the second feature data into the pooling layer to obtain pooling data; A region probability determination unit, used for inputting the pooled data into the fully connected layer to obtain a region category probability; A splicing unit, used for performing feature splicing on each of the first feature data, the pooled data and the regional category probability to obtain spliced ​​feature data; A type probability determination unit, used for inputting each of the spliced ​​feature data into the third perceptron to obtain a predicted type probability; The category prediction unit is used to determine the training prediction category of each of the training point cloud data according to each of the prediction type probabilities.

6. A computer device, characterized in that: The device comprises: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the target detection method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the target detection method as described in any one of claims 1 to 4 is implemented.