A target detection method and device, electronic equipment and readable storage medium

Abstract

The application provides a target detection method and device, electronic equipment and a readable storage medium. The method comprises: acquiring a current frame point cloud and a previous frame point cloud; using a target detection algorithm based on deep learning to perform target detection on the current frame point cloud to obtain a target detection result; performing clustering operation on the current frame point cloud to obtain a clustering result; tracking a third target of the previous frame point cloud to obtain a tracking result of each third target; matching the tracking result with each first target to determine whether there is a first target belonging to the same object as the tracking result; if there is, outputting information of the tracking result; if there is not, matching the tracking result with each second target to determine whether there is a second target belonging to the same object as the tracking result; if there is, outputting information of the tracking result. The application solves the problem of missed detection of a detection method based on deep learning due to insufficient samples.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a target detection method, apparatus, electronic device and readable storage medium. Background Technology

[0002] In the field of LiDAR, deep learning-based target detection algorithms rely on a backbone network to classify objects in point cloud data and obtain their position, size, and heading angle. Commonly used target detection algorithms include SECOND, Voxel-R-CNN, and PointPillars.

[0003] However, deep learning-based object detection methods are greatly affected by the training set. If the training set is not rich enough, the trained deep learning network is prone to missing detections in actual detection. That is, objects in the point cloud data are not detected, resulting in missed detections. Summary of the Invention

[0004] This application provides a target detection method, apparatus, electronic device, and readable storage medium, which can solve the problem of missed detections that may occur in deep learning-based target detection methods due to insufficient training set samples.

[0005] In a first aspect, embodiments of this application provide a target detection method, including:

[0006] Obtain the point cloud of the current frame and the point cloud of the previous frame;

[0007] The point cloud of the current frame is subjected to target detection using a deep learning-based target detection algorithm to obtain target detection results, which include information about each first target.

[0008] Clustering operations are performed on the current frame point cloud to obtain clustering results, which include clusters of each second target;

[0009] The third target of the point cloud in the previous frame is tracked to obtain the tracking results of each third target, wherein the third target is an object with a confidence level greater than or equal to a first confidence level threshold;

[0010] For each of the tracking results of the third target, the tracking results are matched with each of the first targets to determine whether there is a first target in the target detection results that belongs to the same object as the tracking results;

[0011] If the first target exists as a single object, output the information of the first target.

[0012] If no first target belonging to the same object exists, the tracking result is matched with each of the second targets to determine whether there is a second target belonging to the same object as the tracking result in the clustering result;

[0013] If a second target belonging to the same object exists, the tracking result information is output.

[0014] Optionally, the information about the first target includes confidence level;

[0015] The step of using a deep learning-based target detection algorithm to detect targets in the current frame point cloud and obtaining target detection results includes:

[0016] Using the aforementioned target detection algorithm, target detection is performed on the current frame point cloud to obtain the target detection result;

[0017] In the target detection results, a first high-confidence target is selected, which is a first target whose confidence level is greater than or equal to the first confidence threshold;

[0018] Filter the second high-confidence target, which is the first target whose confidence level is greater than or equal to the second confidence threshold, and the second confidence threshold is less than the first confidence threshold;

[0019] The first target includes the first high-confidence target and the second high-confidence target.

[0020] Optionally, matching the tracking results with each of the first targets to determine whether there is a first target in the target detection results that belongs to the same object as the tracking results includes:

[0021] Calculate the first intersection-union ratio (IUU) between the tracking results and each of the first high-confidence targets;

[0022] If the first intersection-union ratio is greater than or equal to a preset intersection-union ratio threshold, then it is determined that there exists a first target belonging to the same object as the tracking result;

[0023] If all of the first cross-union ratios are less than the preset cross-union ratio threshold, then calculate the second cross-union ratio between the tracking result and each of the second high-confidence targets;

[0024] If the second cross-union ratio is greater than or equal to the preset cross-union ratio threshold, then it is determined that there is a first target belonging to the same object as the tracking result;

[0025] Update the confidence level of the object to which the second high-confidence target belongs, where the latest confidence level of the object to which the second high-confidence target belongs is greater than the first confidence level threshold;

[0026] If all of the second cross-union ratios are less than the preset cross-union ratio threshold, then it is determined that there is no first target belonging to the same object as the tracking result.

[0027] Optionally, after determining whether there exists a second target in the clustering results that belongs to the same object as the tracking results, the method further includes:

[0028] If there is no second target belonging to the same object, then update the trajectory parameter value.

[0029] Optionally, matching the tracking results with each of the second targets to determine whether there exists a second target in the clustering results that belongs to the same object as the tracking results includes:

[0030] Calculate the third intersection-union ratio (IUU) between the tracking results and each of the second targets;

[0031] If the third cross-union ratio is greater than or equal to the preset cross-union ratio threshold, then it is determined that there is a second target belonging to the same object as the tracking result;

[0032] Update the confidence level of the tracking result, wherein the latest confidence level of the tracking result is greater than the first confidence level threshold;

[0033] If all of the third cross-union ratios are less than the preset cross-union ratio threshold, then it is determined that there is no second target belonging to the same object as the tracking result.

[0034] Optionally, after updating the trajectory parameter values, the method further includes:

[0035] If the updated trajectory parameter value is less than the preset trajectory parameter value, then the next frame point cloud is obtained as the current frame point cloud, and the process proceeds to the step: using a deep learning-based target detection algorithm to perform target detection on the current frame point cloud to obtain the target detection result;

[0036] If the updated trajectory parameter value is greater than or equal to the preset trajectory parameter value, then the trajectory corresponding to the tracking result is deleted.

[0037] Optionally, the first confidence threshold is determined based on the correspondence between the confidence level and the precision and recall.

[0038] Secondly, embodiments of this application provide a target detection device, comprising:

[0039] The acquisition unit is used to acquire the point cloud of the current frame and the point cloud of the previous frame;

[0040] A point cloud processing unit is used to perform target detection on the current frame point cloud using a deep learning-based target detection algorithm to obtain target detection results, the target detection results including information of each first target;

[0041] Used to perform clustering operations on the current frame point cloud to obtain clustering results, the clustering results including clusters of each second target;

[0042] Used to track the third target in the point cloud of the previous frame, and obtain the tracking results of each third target, wherein the third target is an object with a confidence level greater than or equal to a first confidence level threshold;

[0043] A matching unit is configured to match the tracking result of each of the third targets with each of the first targets to determine whether there is a first target in the target detection result that belongs to the same object as the tracking result;

[0044] If a first target belonging to the same object exists, then output the information of the first target;

[0045] If no first target belonging to the same object exists, the tracking result is matched with each of the second targets to determine whether there is a second target belonging to the same object as the tracking result in the clustering result;

[0046] Information used to output the tracking results if a second target belonging to the same object exists.

[0047] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of the first aspects above.

[0048] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0049] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0050] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0051] The beneficial effects of the embodiments in this application compared with the prior art are:

[0052] This application embodiment utilizes a deep learning-based target detection algorithm to perform target detection on the current frame point cloud, obtaining target detection results, which include information on each first target; it tracks the third target in the previous frame point cloud, obtaining tracking results for each third target, where the third target is an object with a confidence level greater than a first confidence threshold; for each third target tracking result, the tracking result is matched with each first target to determine whether there is a first target in the target detection result that belongs to the same object as the tracking result; by combining the tracking result and the target detection result, the state of the object in the current frame point cloud can be accurately obtained, while reducing false detections of objects in the current frame point cloud.

[0053] Clustering operations are performed on the point cloud of the current frame to obtain clustering results, which include clusters of each secondary target. Because clustering operations aggregate points in each cluster to detect objects, they can detect objects that are missed by deep learning-based target detection algorithms.

[0054] By matching the tracking results with each second target, it is determined whether there is a second target in the clustering results that belongs to the same object as the tracking results; if there is a second target that belongs to the same object, the tracking results information is output, which can obtain the information of the missed targets and improve the recall rate of the deep learning-based target detection algorithm. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of the first flowchart of a target detection method provided in an embodiment of this application;

[0057] Figure 2 This is a schematic diagram illustrating the correspondence provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of a second flowchart of a target detection method provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the target detection device provided in one embodiment of this application;

[0060] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0062] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0063] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0064] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0065] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0066] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0067] Figure 1 This is a schematic diagram of the first flowchart of a target detection method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0068] S11: Obtain the point cloud of the current frame and the point cloud of the previous frame.

[0069] In the application, radar scans the surrounding environment to obtain the point cloud at the current moment, thus obtaining the point cloud of the current frame. It also obtains the point cloud from the previous moment, thus obtaining the point cloud of the previous frame. The point cloud includes at least one object.

[0070] S12: Use a deep learning-based target detection algorithm to perform target detection on the point cloud of the current frame and obtain the target detection results.

[0071] The target detection results include information about each primary target.

[0072] In applications, high-performance 3D object detection networks such as Voxel-R-CNN and PointPillars can be used to perform object detection on the point cloud of the current frame and obtain the object detection results.

[0073] The first target information consists of object information obtained from the point cloud using a deep learning-based object detection algorithm. This information includes the target bounding box category, center coordinates, dimensions, heading angle, and confidence score. The bounding box contains the object.

[0074] S13: Perform clustering operations on the point cloud of the current frame to obtain the clustering results.

[0075] The clustering results include clusters for each secondary objective.

[0076] In the application, the clusters of the second objective are the clusters of objects obtained in the point cloud through a clustering algorithm.

[0077] Clustering algorithms such as k-means and DBSCAN can be used to group points in the current frame's point cloud that have a spacing smaller than the cluster radius to form clusters. The number of points in each cluster must meet the minimum number of points required for clustering to obtain the clustering result. Due to the characteristics of clustering algorithms, objects in the point cloud can be accurately detected.

[0078] S14: Track the third target in the point cloud of the previous frame and obtain the tracking results of each third target.

[0079] The third target is an object with a confidence level greater than or equal to the first confidence threshold. The tracking result is a representation of the third target in the point cloud of the current frame, providing a basis for matching with the first and second targets. Information in the tracking result includes the target bounding box category, the center coordinates of the target bounding box, the size information of the target bounding box, the heading angle, and the confidence level. The target bounding box contains the object.

[0080] In applications, the first confidence threshold is used to identify objects that are real and of interest to the user.

[0081] The third target in the point cloud of the previous frame can be tracked using a Kalman filter and a Hungarian matching algorithm to obtain the tracking results, which provides a basis for determining whether there are any missed targets in the target detection results.

[0082] The first confidence threshold is determined based on the correspondence between confidence, precision, and recall. Specifically, the confidence levels corresponding to high precision and high recall are set as the first confidence level. Figure 2 This is a schematic diagram illustrating the correspondence provided in an embodiment of this application. For example... Figure 2 As shown, precision increases with increasing confidence, while recall decreases with increasing confidence. The graph also shows that the optimal confidence level is 0.39, corresponding to relatively high precision and recall.

[0083] S15: For each third target tracking result, match the tracking result with each first target to determine whether there is a first target in the target detection result that belongs to the same object as the tracking result.

[0084] In the application, the tracking result is matched with each first target. If the tracking result matches one of the first targets, it is determined that the tracking result and the first target belong to the same object. If the tracking result does not match any of the first targets, it is determined that there is no first target belonging to the same object as the tracking result.

[0085] S16: If there exists a first target belonging to the same object, output the information of the first target.

[0086] In application, when it is determined that a first target belonging to the same object as the tracked result exists, the first target in the current frame point cloud can be taken as the accurate result. The information of the first target is output as the detection result of that object in the current frame point cloud.

[0087] S17: If there is no first target belonging to the same object, then match the tracking results with each second target to determine whether there is a second target belonging to the same object as the tracking results in the clustering results.

[0088] In applications, if it is determined that there is no first target belonging to the same object as the tracking result, it indicates that the deep learning-based object detection algorithm may have missed detections, and it is necessary to confirm whether there are any missed objects.

[0089] The tracking result is matched against each of the second targets. If the tracking result matches one of the second targets, it is determined that the tracking result and the second target belong to the same object. If the tracking result does not match any of the second targets, it is determined that there is no second target belonging to the same object as the tracking result.

[0090] S18: If a second target belonging to the same object exists, output the tracking result information.

[0091] In the application, the existence of a second target belonging to the same object indicates that there is indeed a missed object in the current frame point cloud, and that the missed object is the second target. However, because the second target is a clustering result, the corresponding object information is inaccurate. The tracking result information is output as the detection result of this object in the current frame point cloud, thus obtaining the detection result of the missed object in the current frame point cloud.

[0092] Understandably, deep learning-based object detection algorithms are heavily influenced by the dataset; insufficient data can lead to missed detections. Clustering algorithms, on the other hand, group points with a distance smaller than the cluster radius to form clusters, ensuring that each cluster meets the minimum number of points required for clustering, thus guaranteeing the detection of all objects in the point cloud. This provides a basis for identifying any missed objects.

[0093] Based on the target detection results, combined with the tracking results obtained by the tracking algorithm and the clustering results obtained by the clustering algorithm, the missed objects can be found, and the detection results of all objects in the current frame point cloud can be obtained, thus improving the recall rate of the deep learning-based target detection algorithm.

[0094] This embodiment utilizes a deep learning-based target detection algorithm to detect targets in the current frame point cloud, obtaining target detection results, which include information about each first target. It then tracks the third targets in the previous frame point cloud, obtaining tracking results for each third target. A third target is an object with a confidence level greater than a first confidence threshold. For each third target's tracking result, the tracking result is matched with each first target to determine if there is a first target in the target detection result that belongs to the same object as the tracked target. By combining the tracking result and the target detection result, the state of the object in the current frame point cloud can be accurately obtained, while also reducing false detections in the current frame point cloud.

[0095] Clustering operations are performed on the point cloud of the current frame to obtain clustering results, which include clusters of each secondary target. Because clustering operations aggregate points in each cluster to detect objects, they can detect objects that are missed by deep learning-based target detection algorithms.

[0096] By matching the tracking results with each second target, it is determined whether there is a second target in the clustering results that belongs to the same object as the tracking results; if there is a second target that belongs to the same object, the tracking results information is output, which can obtain the information of the missed targets and improve the recall rate of the deep learning-based target detection algorithm.

[0097] In one embodiment, the information for the first objective includes a confidence level;

[0098] Step S12 includes:

[0099] S121: Use the target detection algorithm to perform target detection on the point cloud of the current frame and obtain the target detection result.

[0100] S122: In the target detection results, select the first high-confidence target, which is the first target with a confidence level greater than or equal to the first confidence threshold.

[0101] In the application, the target detection results are filtered using a first confidence threshold. Targets with a confidence level greater than or equal to the first confidence threshold are selected as the first high-confidence targets.

[0102] S123: Filter the second highest confidence target. The second highest confidence target is the first target with a confidence level greater than or equal to the second confidence threshold, and the second confidence threshold is less than the first confidence threshold.

[0103] The first objective includes a first high-confidence objective and a second high-confidence objective.

[0104] In the application, after selecting the first high-confidence target, the results are filtered using a second confidence threshold. Targets with confidence levels below the first threshold and greater than or equal to the second threshold are selected as the second high-confidence targets.

[0105] The first target in the target detection results is filtered by using a first confidence threshold and a second confidence threshold to filter out real objects that are of interest to the user.

[0106] Step S15 includes:

[0107] S151: Calculate the first crossover ratio (CRO) between the tracking results and each of the first high-confidence targets.

[0108] In the application, the first intersection-union ratio (IUR) of the target box of the tracking result and the target box of the first high-confidence target is calculated by dividing the area of ​​the overlapping part of the two target boxes by the area of ​​the two targets combined.

[0109] S152: If there exists a first crossover ratio greater than or equal to a preset crossover ratio threshold, then it is determined that there exists a first target belonging to the same object as the tracking result.

[0110] In application, if the first intersection-union ratio (IUR) of the tracking result with one of the first high-confidence targets is greater than or equal to a preset IUR threshold, it indicates that the tracking result matches the first high-confidence target and belongs to the same object. Correspondingly, it is determined that there exists a first target belonging to the same object as the tracking result. The information of the first high-confidence target is output as the detection result of that object in the point cloud of the current frame.

[0111] S153: If all first cross-union ratios are less than the preset cross-union ratio threshold, then calculate the second cross-union ratio between the tracking result and each second high-confidence target.

[0112] In the application, the first intersection-union ratio (IU) of the tracking result with each of the first high-confidence targets is less than the preset IU threshold, indicating that the tracking result does not match each of the first high-confidence targets. Then, the tracking result is matched with each of the second high-confidence targets, and the second IU of the tracking result with each of the second high-confidence targets is calculated to determine the second high-confidence targets that belong to the same object as the tracking result.

[0113] S154: If there exists a second cross-union ratio greater than or equal to the preset cross-union ratio threshold, then it is determined that there exists a first target belonging to the same object as the tracking result.

[0114] In application, if the second intersection-union ratio (IUR) of the tracking result with one of the second high-confidence targets is greater than or equal to a preset IUR threshold, it indicates that the tracking result matches the second high-confidence target and belongs to the same object. Correspondingly, it is determined that there exists a first target belonging to the same object as the tracking result. The information of the second high-confidence target is output as the detection result of that object in the point cloud of the current frame.

[0115] S155: Update the confidence level of the object to which the second highest confidence target belongs. The latest confidence level of the object to which the second highest confidence target belongs is greater than the first confidence level threshold.

[0116] In application, the confidence level of the object to which the second highest confidence target belongs is increased so that it can be used as the target for subsequent tracking algorithms.

[0117] S156: If all second cross-union ratios are less than the preset cross-union ratio threshold, then it is determined that there is no first target belonging to the same object as the tracking result.

[0118] In the application, the second intersection-union ratio (IUR) of the tracking result with each of the second highest confidence targets was less than the preset IUR threshold, indicating that the tracking result did not match each of the second highest confidence targets, and correspondingly, there was no first target belonging to the same object as the tracking result. This suggests that the deep learning-based target detection algorithm may have missed detections.

[0119] By first matching the tracking result with the first high-confidence target, and only after failing to match the first high-confidence target, matching it with the second high-confidence target can reduce the time required to match the tracking result with the first target.

[0120] Step S17 includes:

[0121] S171: Calculate the third intersection-union ratio between the tracking results and each second target.

[0122] S172: If there is a third cross-union ratio greater than or equal to the preset cross-union ratio threshold, then it is determined that there is a second target belonging to the same object as the tracking result.

[0123] In application, if the intersection-union ratio (IU) of the tracking result with one of the second targets is greater than or equal to a preset IU threshold, it indicates that the tracking result matches the second target and belongs to the same object. This confirms the existence of a second target belonging to the same object as the tracking result. This demonstrates that deep learning-based target detection algorithms do indeed have instances of missed detections. The tracking result information can be output as the detection result of the missed target in the current frame's point cloud. The tracking result obtained through the tracking algorithm can compensate for the inaccuracy of target information in the clustering results, thus obtaining the detection result of the missed target in the current frame's point cloud.

[0124] S173: Update the confidence level of the tracking results. The latest confidence level of the tracking results is greater than the first confidence level threshold.

[0125] In applications, the confidence level of the tracking results is increased so that the objects in the target bounding box of the tracking results can be used as the execution objects of the tracking algorithm in subsequent applications.

[0126] S174: If all third cross-union ratios are less than the preset cross-union ratio threshold, then it is determined that there is no second target belonging to the same object as the tracking result.

[0127] In the application, the third intersection-union ratio (IU) of the tracking result with each second target was less than the preset IU threshold, indicating that the tracking result did not match each second target, and thus it was determined that there was no second target belonging to the same object as the tracking result. This shows that the deep learning-based target detection algorithm may not have missed any detections.

[0128] Figure 3 This is a schematic diagram of a second flowchart of a target detection method provided in an embodiment of this application. For example... Figure 3 As shown, after step S17, the following steps are also included:

[0129] S19: If there is no second target belonging to the same object, update the trajectory parameter value.

[0130] In the application, although there may be no second target belonging to the same object, and therefore no missed detections are possible, the tracking result and its trajectory are retained to ensure that no missed detections are indeed found. This is for subsequent tracking to verify whether the object in the target box of the tracking result is indeed the missed object.

[0131] If an object is not among the missed detections, but the tracking result and its trajectory are continuously retained and tracked, it will cause detection problems. Setting trajectory parameters, whose values ​​characterize the tracking status of the object in the tracking result, is crucial for determining whether the object truly belongs to the missed detections.

[0132] S20: If the updated trajectory parameter value is less than the preset trajectory parameter value, then obtain the next frame point cloud as the current frame point cloud, and proceed to the step: use a deep learning-based target detection algorithm to perform target detection on the current frame point cloud to obtain the target detection result.

[0133] In the application, if the updated trajectory parameter value is less than the preset trajectory parameter value, it indicates that the number of unmatched frames for the object during continuous tracking has not reached the preset threshold. Therefore, it cannot be determined whether the object in the target box of the tracking result is a missed detection. Thus, tracking continues, and at the next moment, the surrounding environment is scanned by radar to obtain the point cloud for the next moment, thus acquiring the next frame point cloud. The point cloud of the next frame at the next moment is used as the current frame point cloud, and the point cloud of the current frame at the current moment is used as the previous frame point cloud, and target detection continues.

[0134] S21: If the updated trajectory parameter value is greater than or equal to the preset trajectory parameter value, then delete the trajectory corresponding to the tracking result.

[0135] In the application, if the updated trajectory parameter value is greater than or equal to the preset trajectory parameter value, it indicates that the number of frames in which the object is not matched has reached the preset threshold while continuously tracking the object. At this point, it can be determined that the object in the target box of the tracking result is not a missed object, and tracking will stop. Therefore, the trajectory is deleted.

[0136] For example, in the first frame, there is a third target A1 (object ID is 1), and the confidence level of the third target is greater than or equal to the first confidence threshold. The tracking algorithm tracks the third target A1 in the first frame. The tracking result of the third target A1 in the second frame is T2. The tracking result T2 is matched with the target in the second frame. If there is a target that matches the tracking result T2, the result of the second frame is output, which is either the first target A2 or the tracking result T2. If there is no target that matches the tracking result T2, the trajectory parameter value of the object with ID 1 is incremented by 1 (initially 0), and the updated trajectory parameter value is 1. The result of the second frame is then output: tracking result T2. Here, the updated trajectory parameter value of 1 indicates that the object with ID 1 has not been matched in 1 frame.

[0137] Assume there is no target matching the tracking result T2 in the second frame, so the trajectory parameter value is 1. Tracking result T2 is the third target. The tracking algorithm is used to track the tracking result T2 in the second frame, resulting in tracking result T3 in the third frame. The tracking result T3 is then matched against the target in the second frame. If a target matches the tracking result T3, the result of the third frame is output, which is either the first target A3 or the tracking result T3, and the trajectory parameter value of the object with ID 1 is set to 0. If no target matches the tracking result T3, the trajectory parameter value of the object with ID 1 is incremented by 1, resulting in an updated trajectory parameter value of 2, indicating that the object with ID 1 has not been matched for 2 frames.

[0138] Continue tracking the object with ID 1 according to the above steps until the updated trajectory parameter value is greater than the preset trajectory parameter value. When the updated trajectory parameter value is greater than the preset trajectory parameter value, it is determined that the object with ID 1 is not a missed object, the trajectory of the object with ID 1 is deleted, and tracking is stopped.

[0139] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0140] For ease of explanation, only the parts related to the embodiments of this application are shown in the methods described in the above embodiments.

[0141] Figure 4 This is a schematic diagram of the target detection device provided in one embodiment of this application. Figure 4 As shown, the device includes:

[0142] Acquisition unit 10 is used to acquire the point cloud of the current frame and the point cloud of the previous frame;

[0143] Point cloud processing unit 11 is used to perform target detection on the current frame point cloud using a deep learning-based target detection algorithm to obtain target detection results, which include information of each first target.

[0144] Used to perform clustering operations on the point cloud of the current frame to obtain clustering results, which include clusters of each secondary target;

[0145] Used to track the third target in the point cloud of the previous frame, and obtain the tracking results of each third target. The third target is an object with a confidence level greater than or equal to the first confidence threshold.

[0146] The matching unit 12 is used to match the tracking results of each third target with each first target to determine whether there is a first target in the target detection results that belongs to the same object as the tracking results;

[0147] If a first target belonging to the same object exists, the information of the first target will be output.

[0148] If no first target belonging to the same object exists, the tracking result is matched with each second target to determine whether there is a second target belonging to the same object as the tracking result in the clustering result.

[0149] This is used to output tracking results if a second target belonging to the same object exists.

[0150] In one embodiment, the point cloud processing unit is specifically used to perform target detection on the point cloud of the current frame using a target detection algorithm to obtain the target detection result;

[0151] In the target detection results, the first high-confidence target is selected. The first high-confidence target is the first target with a confidence level greater than or equal to the first confidence threshold.

[0152] Select the second highest confidence target. The second highest confidence target is the first target with a confidence level greater than or equal to the second confidence threshold, and the second confidence threshold is less than the first confidence threshold.

[0153] The first objective includes a first high-confidence objective and a second high-confidence objective.

[0154] In one embodiment, the matching unit is specifically used to calculate a first intersection-union ratio (IU) between the tracking results and each of the first high-confidence targets;

[0155] If there exists a first intersection-union ratio greater than or equal to a preset intersection-union ratio threshold, then it is determined that there exists a first target belonging to the same object as the tracking result;

[0156] If all first cross-union ratios are less than the preset cross-union ratio threshold, then the second cross-union ratio between the tracking results and each second high-confidence target is calculated;

[0157] If there exists a second intersection-union ratio greater than or equal to a preset intersection-union ratio threshold, then it is determined that there exists a first target belonging to the same object as the tracking result;

[0158] Update the confidence level of the object to which the second highest confidence target belongs. The latest confidence level of the object to which the second highest confidence target belongs is greater than the first confidence threshold.

[0159] If the second crossover ratio is less than the preset crossover ratio threshold, then it is determined that there is no first target belonging to the same object as the tracking result.

[0160] In one embodiment, the matching unit is specifically used to calculate the third intersection-union ratio between the tracking result and each of the second targets.

[0161] If a third cross-union ratio (CUNR) is greater than or equal to a preset CUNR threshold, then it is determined that there is a second target belonging to the same object as the tracking result;

[0162] Update the confidence level of the tracking results; the latest confidence level of the tracking results is greater than the first confidence level threshold.

[0163] If all third cross-union ratios are less than the preset cross-union ratio threshold, then it is determined that there is no second target belonging to the same object as the tracking result;

[0164] In one embodiment, the matching unit is further configured to update the trajectory parameter value if there is no second target belonging to the same object.

[0165] In one embodiment, the matching unit is further configured to, if the updated trajectory parameter value is less than the preset trajectory parameter value, obtain the next frame point cloud as the current frame point cloud and proceed to the step: using a deep learning-based target detection algorithm to perform target detection on the current frame point cloud and obtain the target detection result;

[0166] If the updated trajectory parameter value is greater than or equal to the preset trajectory parameter value, then the trajectory corresponding to the tracking result is deleted.

[0167] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 2 of this embodiment includes: at least one processor 20 ( Figure 5 (Only one is shown in the diagram), memory 21, and computer program 22 stored in said memory 21 and executable on said at least one processor 20, wherein said processor 20 executes said computer program 22 to implement the steps in any of the above method embodiments.

[0168] The electronic device 2 can be a desktop computer, laptop, handheld computer, or vehicle-mounted device. The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 2 and does not constitute a limitation on electronic device 2. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0169] The processor 20 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0170] In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or memory of the electronic device 2. In other embodiments, the memory 21 may be an external storage device of the electronic device 2, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 2. Furthermore, the memory 21 may include both internal and external storage units of the electronic device 2. The memory 21 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0171] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0174] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0175] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0176] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0177] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0179] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target detection method characterized by, The method comprises the following steps: obtaining a current frame point cloud and a previous frame point cloud; performing target detection on the current frame point cloud by using a target detection algorithm based on deep learning to obtain a target detection result, the target detection result comprising information of each first target; performing clustering operation on the current frame point cloud to obtain a clustering result, the clustering result comprising a clustering cluster of each second target; tracking a third target of the previous frame point cloud to obtain a tracking result of each third target, the third target being an object with a confidence greater than or equal to a first confidence threshold; for each tracking result of the third target, matching the tracking result with each first target to determine whether there is a first target belonging to the same object as the third target in the target detection result; if there is a first target belonging to the same object, outputting information of the first target; if there is no first target belonging to the same object, matching the tracking result with each second target to determine whether there is a second target belonging to the same object as the tracking result in the clustering result; if there is a second target belonging to the same object, outputting information of the tracking result.

2. The method of claim 1, wherein, The information of the first target comprises a confidence; the target detection on the current frame point cloud by using the target detection algorithm based on deep learning to obtain the target detection result comprises the following steps: performing target detection on the current frame point cloud by using the target detection algorithm to obtain a target detection result; in the target detection result, screening a first high-confidence target, the first high-confidence target being a first target with a confidence greater than or equal to the first confidence threshold; screening a second high-confidence target, the second high-confidence target being a first target with a confidence greater than or equal to a second confidence threshold, the second confidence threshold being less than the first confidence threshold; wherein the first target comprises the first high-confidence target and the second high-confidence target.

3. The method of claim 2, wherein, the matching of the tracking result with each first target to determine whether there is a first target belonging to the same object as the tracking result in the target detection result comprises the following steps: calculating a first intersection-over-union between the tracking result and each first high-confidence target; if there is a first intersection-over-union greater than or equal to a preset intersection-over-union threshold, it is determined that there is a first target belonging to the same object as the tracking result; if all the first intersection-over-union is less than the preset intersection-over-union threshold, calculating a second intersection-over-union between the tracking result and each second high-confidence target; if there is a second intersection-over-union greater than or equal to the preset intersection-over-union threshold, it is determined that there is a first target belonging to the same object as the tracking result; updating the confidence of the object to which the second high-confidence target belongs, the latest confidence of the object to which the second high-confidence target belongs being greater than the first confidence threshold; if all the second intersection-over-union is less than the preset intersection-over-union threshold, it is determined that there is no first target belonging to the same object as the tracking result.

4. The method of claim 1, wherein, The method further comprises, after the determining whether the second target belonging to the same object exists in the clustering result: If the second target belonging to the same object does not exist, updating the track parameter value.

5. The method of claim 4, wherein, The matching the tracking result with each second target, and determining whether the second target belonging to the same object exists in the clustering result, comprises: Calculating a third intersection-over-union of the tracking result and each second target; If the third intersection-over-union is greater than or equal to a preset intersection-over-union threshold, it is determined that the second target belonging to the same object exists; Updating the confidence of the tracking result, and the latest confidence of the tracking result is greater than the first confidence threshold; If all the third intersection-over-unions are less than the preset intersection-over-union threshold, it is determined that the second target belonging to the same object does not exist.

6. The method of claim 4 or 5, wherein, The method further comprises, after the updating the track parameter value: If the updated track parameter value is less than a preset track parameter value, a next frame point cloud is obtained as the current frame point cloud, and the step of performing target detection on the current frame point cloud by using the target detection algorithm based on deep learning to obtain a target detection result is entered; If the updated track parameter value is greater than or equal to the preset track parameter value, a track corresponding to the tracking result is deleted.

7. The method of claim 1, wherein: The first confidence threshold is determined according to a corresponding relationship between the confidence and the accuracy and the recall rate.

8. A target detection apparatus characterized by comprising: The method comprises: An acquisition unit is configured to acquire a current frame point cloud and a previous frame point cloud; A point cloud processing unit is configured to perform target detection on the current frame point cloud by using a target detection algorithm based on deep learning to obtain a target detection result, the target detection result comprising information of each first target; The current frame point cloud is subjected to clustering operation to obtain a clustering result, the clustering result comprising a clustering cluster of each second target; Each third target of the previous frame point cloud is tracked to obtain a tracking result of each third target, the third target being an object with a confidence greater than or equal to a first confidence threshold; A matching unit is configured to, for the tracking result of each third target, match the tracking result with each first target to determine whether the first target belonging to the same object exists in the target detection result; If the first target belonging to the same object exists, information of the first target is outputted; If the first target belonging to the same object does not exist, the tracking result is matched with each second target to determine whether the second target belonging to the same object exists in the clustering result; If the second target belonging to the same object exists, information of the tracking result is outputted.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images