Multi-detector result fusion tracking method, device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses a multi-detector result fusion tracking method, device, equipment and storage medium, which are used to improve the accuracy and precision of obstacle identification and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving. The multi-detector result fusion tracking method includes: obtaining obstacle historical trajectory information, first obstacle information and second obstacle information, the first obstacle information and the second obstacle information are derived from different types of detectors; obstacle matching the first obstacle information and the second obstacle information with the obstacle historical trajectory information respectively through a preset multi-target tracking association module to obtain a first obstacle matching result and a second obstacle matching result; obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-detector result fusion tracking method, device, equipment and storage medium. Background Art

[0002] In the environmental perception of autonomous driving, multiple sensors are usually used to identify obstacles, and each sensor has a dedicated detector so that the downstream tracking module has more information when performing trajectory matching.

[0003] In the prior art, multiple detectors are used to detect obstacles simultaneously. There are differences in the confidence levels of the obstacle recognition results of different detectors, making it difficult to determine the accuracy of the obstacle recognition results of each detector. For example, deep learning detectors perform obstacle recognition in a data-driven manner, which has better accuracy than rule-based strategy detectors, but there is a situation where obstacles are missed, resulting in low accuracy in obstacle recognition and obstacle trajectory construction. Summary of the invention

[0004] The present invention provides a multi-detector result fusion tracking method, device, equipment and storage medium for improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0005] To achieve the above-mentioned objective, a first aspect of the present invention provides a multi-detector result fusion tracking method, comprising: obtaining obstacle historical trajectory information, first obstacle information and second obstacle information, wherein the first obstacle information and the second obstacle information are derived from different types of detectors; performing obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result; and dividing obstacle matching results and updating obstacle trajectory status according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information.

[0006] In a feasible implementation manner, obtaining obstacle historical trajectory information, first obstacle information, and second obstacle information includes: receiving a multi-detector result fusion tracking request, and performing an authority verification process on the multi-detector result fusion tracking request to obtain an authority verification result; when the authority verification result is a verification pass, reading the obstacle historical trajectory information, first obstacle information, and second obstacle information from a preset database according to the multi-detector result fusion tracking request.

[0007] In a feasible implementation manner, the performing obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result includes: performing obstacle association processing on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data; performing obstacle matching processing on the first obstacle association data and the second obstacle association data respectively according to a preset matching algorithm to obtain a first obstacle matching result and a second obstacle matching result.

[0008] In a feasible implementation manner, the performing obstacle association processing on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data includes: performing feature extraction on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle feature data, second obstacle feature data, and obstacle historical trajectory feature data; and performing correlation analysis on the first obstacle feature data and the second obstacle feature data with the obstacle historical trajectory feature data, respectively, to obtain first obstacle association data and second obstacle association data.

[0009] In a feasible implementation manner, the dividing obstacle matching results and updating the obstacle trajectory status according to the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information includes: dividing the obstacle matching results of the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information by using a preset matching check model to obtain an obstacle division result, wherein the obstacle division result includes a first current frame obstacle that only successfully matches the first obstacle information, a second current frame obstacle that only successfully matches the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information; and performing confidence processing and updating the obstacle trajectory status according to the obstacle division result and the obstacle historical trajectory information.

[0010] In a feasible implementation manner, the performing confidence processing and obstacle trajectory state updating according to the obstacle division result and the obstacle historical trajectory information includes: when the obstacle division result is the first current frame obstacle, updating the obstacle trajectory state according to the first current frame obstacle and the obstacle historical trajectory information, the obstacle trajectory state including obstacle position, obstacle size, obstacle speed, obstacle acceleration and obstacle type; when the obstacle division result is the second current frame obstacle, dividing the second current frame obstacle into a low confidence obstacle or a high confidence obstacle according to a preset confidence threshold, and performing an obstacle trajectory state updating operation on the obstacle historical trajectory information according to the low confidence obstacle or the high confidence obstacle; when the obstacle division result is the third current frame obstacle, increasing the confidence corresponding to the third current frame obstacle.

[0011] In a feasible implementation manner, after dividing the obstacle matching results and updating the obstacle trajectory status according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, the multi-detector result fusion tracking method further includes: acquiring obstacle information that has not been successfully matched with the obstacle historical trajectory information; generating new obstacle trajectory information according to the obstacle information that has not been successfully matched with the obstacle historical trajectory information, and performing trajectory publishing processing on the new obstacle trajectory information according to a preset publishing rule.

[0012] A second aspect of the present invention provides a multi-detector result fusion tracking device, including: a first acquisition module, used to acquire obstacle historical trajectory information, first obstacle information and second obstacle information, wherein the first obstacle information and the second obstacle information come from different types of detectors; a matching module, used to perform obstacle matching on the first obstacle information and the second obstacle information with the obstacle historical trajectory information respectively to obtain a first obstacle matching result and a second obstacle matching result; and a division module, used to divide the obstacle matching results and update the obstacle trajectory status according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information.

[0013] In a feasible implementation manner, the first acquisition module is specifically used to: receive a multi-detector result fusion tracking request, and perform an authority verification process on the multi-detector result fusion tracking request to obtain an authority verification result; when the authority verification result is a passed verification, read the obstacle historical trajectory information, the first obstacle information and the second obstacle information from a preset database according to the multi-detector result fusion tracking request.

[0014] In a feasible implementation manner, the matching module further includes: an association unit, configured to perform obstacle association processing on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data; and a matching unit, configured to perform obstacle matching processing on the first obstacle association data and the second obstacle association data respectively according to a preset matching algorithm to obtain a first obstacle matching result and a second obstacle matching result.

[0015] In a feasible implementation manner, the association unit is specifically used to: perform feature extraction on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle feature data, second obstacle feature data, and obstacle historical trajectory feature data; and perform correlation analysis on the first obstacle feature data and the second obstacle feature data with the obstacle historical trajectory feature data, respectively, to obtain first obstacle association data and second obstacle association data.

[0016] In a feasible implementation manner, the division module further includes: a division unit, configured to divide the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information by using a preset matching check model to obtain an obstacle division result, wherein the obstacle division result includes a first current frame obstacle that only successfully matches the first obstacle information, a second current frame obstacle that only successfully matches the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information; and a processing unit, configured to perform confidence processing and obstacle trajectory status update according to the obstacle division result and the obstacle historical trajectory information.

[0017] In a feasible implementation manner, the processing unit is specifically configured to: when the obstacle division result is the first current frame obstacle, update the obstacle trajectory state according to the first current frame obstacle and the obstacle historical trajectory information, wherein the obstacle trajectory state includes obstacle position, obstacle size, obstacle speed, obstacle acceleration, and obstacle type; when the obstacle division result is the second current frame obstacle, divide the second current frame obstacle into a low-confidence obstacle or a high-confidence obstacle according to a preset confidence threshold, and perform an obstacle trajectory state update operation on the obstacle historical trajectory information according to the low-confidence obstacle or the high-confidence obstacle; and when the obstacle division result is the third current frame obstacle, increase the confidence corresponding to the third current frame obstacle.

[0018] In a feasible implementation manner, the multi-detector result fusion tracking device also includes: a second acquisition module, used to obtain obstacle information that has not been successfully matched with the obstacle historical trajectory information; a publishing module, used to generate new obstacle trajectory information based on the obstacle information that has not been successfully matched with the obstacle historical trajectory information, and perform trajectory publishing processing on the new obstacle trajectory information according to preset publishing rules.

[0019] The third aspect of the present invention provides a multi-detector result fusion tracking device, comprising: a memory and at least one processor, the memory storing instructions, the memory and the at least one processor being interconnected through lines; the at least one processor calls the instructions in the memory so that the multi-detector result fusion tracking device executes the above-mentioned multi-detector result fusion tracking method.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned multi-detector result fusion tracking method.

[0021] In the technical solution provided by the present invention, obstacle historical trajectory information, first obstacle information and second obstacle information are obtained, and the first obstacle information and the second obstacle information are derived from different types of detectors; obstacle matching is performed on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result; obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information. In the embodiment of the present invention, obstacle matching is performed on obstacle information corresponding to different types of detectors respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result, and obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, thereby improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an embodiment of a multi-detector result fusion tracking method in an embodiment of the present invention;

[0023] Figure 2 It is a schematic diagram of another embodiment of the multi-detector result fusion tracking method in an embodiment of the present invention;

[0024] Figure 3A schematic diagram of an embodiment of a multi-detector result fusion tracking device in an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of another embodiment of a multi-detector result fusion tracking device according to an embodiment of the present invention;

[0026] Figure 5 It is a schematic diagram of an embodiment of a multi-detector result fusion tracking device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present invention provide a multi-detector result fusion tracking method, apparatus, device and storage medium for improving the accuracy and precision of obstacle identification and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0028] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the multi-detector result fusion tracking method in the embodiment of the present invention includes:

[0030] 101. Obtain obstacle historical trajectory information, first obstacle information, and second obstacle information, where the first obstacle information and the second obstacle information come from different types of detectors.

[0031] Among them, different types of detectors (for example, deep learning detectors and rule strategy detectors, etc.) are applied to unmanned driving scenes to detect and identify obstacles. Obstacles are used to indicate objects such as terrain, landforms and engineering facilities that delay or prevent the operation of the unmanned driving subject. The detectors corresponding to the first obstacle information and the second obstacle information have different detection methods. That is, the first obstacle information and the second obstacle information come from different types of detectors. In this embodiment, only two types of detectors are used as examples for explanation. It can also be three types of detectors, which are not limited here. Taking the target detector of laser radar as an example, the server collects and outputs the first obstacle information and the second obstacle information respectively through the deep learning detector and the rule strategy detector. The first obstacle information and the second obstacle information can both include information such as the contour information expressing each obstacle, the prediction box of each obstacle, the position information of each obstacle, the size of each obstacle and the confidence of each obstacle. The obstacle history trajectory information is used to indicate the obstacle trajectory data before the current moment. The obstacle history trajectory information contains the trajectory state of each obstacle at different moments. The trajectory state of each obstacle includes the position of each obstacle, the size of each obstacle, the speed of each obstacle, the acceleration of each obstacle and the type of each obstacle. The length of the obstacle history track information is the duration of the obstacle's existence.

[0032] In some embodiments, the server receives a multi-detector result fusion tracking request, and performs a permission verification process on the multi-detector result fusion tracking request to obtain a permission verification result. Specifically, the server extracts a user identifier from the multi-detector result fusion tracking request, and the server determines whether the user has the operation authority according to the user identifier. If the user has the operation authority, the server determines that the permission verification result is a verification pass. If the user does not have the operation authority, the server determines that the permission verification result is a verification failure, thereby improving the accuracy of the operation authority processing. When the permission verification result is a verification pass, the server reads the obstacle historical trajectory information, the first obstacle information, and the second obstacle information from the preset database according to the multi-detector result fusion tracking request. Further, when the permission verification result is a verification failure, the server generates a prompt message indicating that the multi-detector result fusion tracking request access failed, and returns the prompt message to the target terminal.

[0033] It is understandable that the execution subject of the present invention may be a multi-detector result fusion tracking device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0034] 102. Perform obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result.

[0035] In some embodiments, the server performs obstacle matching on the first obstacle information and the obstacle historical trajectory information through a preset multi-target tracking association module to obtain a first obstacle matching result; the server performs obstacle matching on the second obstacle information and the obstacle historical trajectory information through a preset multi-target tracking association module to obtain a second obstacle matching result.

[0036] It should be noted that the preset multi-target tracking association module includes a preset association model and a preset matching algorithm. For the above two obstacle matching processes, the preset association model in the preset multi-target tracking association module can select the same association model according to the preset business requirements, and the feature dimensions corresponding to the same association model are consistent, and the processing characteristics are also the same; the preset association model in the preset multi-target tracking association module can also set different association models according to the preset business requirements to respectively process the obstacle information corresponding to different types of detectors (that is, the first obstacle information and the second obstacle information). For example, the preset association model in the preset multi-target tracking association module corresponding to the first obstacle information is a trained neural network model, and the preset association model in the preset multi-target tracking association module corresponding to the second obstacle information is a Kalman filter model. The preset matching algorithm can also be set according to the preset business requirements, which is not limited here.

[0037] 103. Perform obstacle matching result division and obstacle track status update according to the first obstacle matching result, the second obstacle matching result, and the obstacle historical track information.

[0038] It should be noted that all trajectories in the obstacle history trajectory information have corresponding unique trajectory identifiers. The first obstacle matching result and the second obstacle matching result in this embodiment are both cross-frame obstacle matching. After the server obtains the current frame obstacles (that is, the first obstacle information and the second obstacle information) output by different types of detectors, the server matches each obstacle in the obstacle history trajectory information with the currently observed obstacle (that is, the current frame obstacle) one by one, thereby determining that the current frame obstacle matches any historical trajectory in the obstacle history trajectory information, and obtaining the first obstacle matching result and the second obstacle matching result, wherein the current frame obstacle has a corresponding unique trajectory identifier. Specifically, the server inputs the first obstacle matching result, the second obstacle matching result and the obstacle history trajectory information into a preset matching check model, and obtains the first current frame obstacle that only successfully matches the first obstacle information, the second current frame obstacle that only successfully matches the second obstacle information, and the third current frame obstacle that successfully matches both types of obstacle information. The two types of obstacle information come from different types of detectors. The obstacle matching results corresponding to different types of detectors have different weights. For example, the first obstacle matching result corresponding to the deep learning detector is mainly used, and the second obstacle matching result corresponding to the rule strategy detector is supplemented, thereby realizing multi-target tracking.

[0039] Furthermore, if both types of obstacle information are matched successfully, the server enhances the confidence of the third current frame obstacle; if only the first obstacle information is matched successfully, the obstacle trajectory state is updated according to the first current frame obstacle; if only the second obstacle information is matched successfully, the second current frame obstacle is divided into a low-confidence obstacle or a high-confidence obstacle according to the preset confidence threshold, and the server extends the life cycle of the target historical trajectory for the low-confidence obstacle; for the high-confidence obstacle, the server predicts the position and size of the high-confidence obstacle according to the obstacle position, obstacle speed, obstacle size and other status information in the target historical trajectory and the status information of the second current frame obstacle, so as to achieve the purpose of recalling potential missed detections. Among them, the target historical trajectory is used to indicate any historical trajectory in the obstacle historical trajectory information that matches the second current frame obstacle.

[0040] In the embodiment of the present invention, obstacle matching is performed on obstacle information corresponding to different types of detectors and obstacle historical trajectory information respectively to obtain a first obstacle matching result and a second obstacle matching result, and obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, thereby improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0041] See also Figure 2 Another embodiment of the multi-detector result fusion tracking method in the embodiment of the present invention includes:

[0042] 201. Obtain obstacle historical trajectory information, first obstacle information, and second obstacle information, where the first obstacle information and the second obstacle information come from different types of detectors.

[0043] The specific execution process of step 201 is similar to the specific execution process of step 101, and will not be repeated here.

[0044] 202. Perform obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result.

[0045] In some embodiments, the server performs obstacle association processing on the first obstacle information, the second obstacle information and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data, wherein the preset association model may be a trained neural network model, a Kalman filter model, or a model based on a reinforcement learning scoring mechanism, which is not specifically limited here; the server performs obstacle matching processing on the first obstacle association data and the second obstacle association data respectively according to a preset matching algorithm to obtain a first obstacle matching result and a second obstacle matching result, wherein the preset matching algorithm may be a Hungarian algorithm, a bipartite graph maximum matching algorithm, or a bipartite graph perfect matching algorithm, which is not specifically limited here. The first obstacle matching result is used to indicate the matching information between the first obstacle information and the obstacle historical trajectory information, and the second obstacle matching result is used to indicate the matching information between the second obstacle information and the obstacle historical trajectory information.

[0046] Further, when the server performs the step of performing obstacle association processing on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data, in some embodiments, the server performs feature extraction on the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through the preset association model to obtain first obstacle feature data, second obstacle feature data, and obstacle historical trajectory feature data, wherein the feature dimensions corresponding to the first obstacle feature data, the second obstacle feature data, and the obstacle historical trajectory feature data are all the same; the server performs correlation analysis on the first obstacle feature data and the second obstacle feature data with the obstacle historical trajectory feature data, respectively, to obtain first obstacle association data and second obstacle association data.

[0047] 203. Perform obstacle matching result division and obstacle track status update according to the first obstacle matching result, the second obstacle matching result, and the obstacle historical track information.

[0048] That is, the server checks whether the obstacle history track information matches the intersection data between the first obstacle matching result and the second obstacle matching result, the server checks whether the obstacle history track information matches only the data other than the intersection data in the first obstacle matching result, and the server checks whether the obstacle history track information matches only the data other than the intersection data in the second obstacle matching result. In some embodiments, the server divides the obstacle matching results of the first obstacle matching result, the second obstacle matching result, and the obstacle history track information by a preset matching check model to obtain an obstacle division result, and the obstacle division result includes a first current frame obstacle that only successfully matches the first obstacle information, a second current frame obstacle that only successfully matches the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information (that is, the first obstacle information and the second obstacle information); the server performs confidence processing and updates the obstacle track status according to the obstacle division result and the obstacle history track information.

[0049] Further, when the server performs the steps of performing confidence processing and updating the obstacle trajectory state according to the obstacle division result and the obstacle historical trajectory information, in some embodiments, when the obstacle division result is a first current frame obstacle, the server updates the obstacle trajectory state according to the first current frame obstacle and the obstacle historical trajectory information, and the obstacle trajectory state includes the obstacle position, obstacle size, obstacle speed, obstacle acceleration and obstacle type; when the obstacle division result is a second current frame obstacle, the server divides the second current frame obstacle into a low confidence obstacle or a high confidence obstacle according to a preset confidence threshold, and updates the obstacle trajectory state according to the low confidence obstacle or the high confidence obstacle. The obstacle history trajectory information performs an obstacle trajectory state update operation, for example, the preset confidence threshold may be 0.65 or 0.5; when the obstacle division result is the third current frame obstacle, the server increases the confidence corresponding to the third current frame obstacle, for example, the server adjusts the confidence corresponding to the third current frame obstacle from 0.44 to 0.484, that is, multiplying the confidence corresponding to the third current frame obstacle by a preset coefficient (for example, the preset coefficient is 1.1), or the server adjusts the confidence corresponding to the third current frame obstacle from 0.44 to 0.55, that is, adding a preset value to the confidence corresponding to the third current frame obstacle (for example, the preset value is 0.11). The value range of the confidence is greater than or equal to 0 and less than or equal to 1. For example, the first current frame obstacle, the second current frame obstacle, and the third current frame obstacle are pedestrians, cars, and tricycles, respectively.

[0050] 204. Obtain obstacle information that has not been successfully matched with the obstacle historical trajectory information.

[0051] Specifically, the server removes the first obstacle matching result from the first obstacle information to obtain first unmatched obstacle information; the server removes the second obstacle matching result from the second obstacle information to obtain second unmatched obstacle information; the server performs data deduplication processing on the first unmatched obstacle information and the second unmatched obstacle information to obtain obstacle information that has not been successfully matched with the obstacle historical trajectory information, and performs data persistence processing on the obstacle information that has not been successfully matched with the obstacle historical trajectory information.

[0052] 205. Generate new obstacle trajectory information according to the obstacle information that has not been successfully matched with the historical obstacle trajectory information, and perform trajectory publishing processing on the new obstacle trajectory information according to a preset publishing rule.

[0053] That is, the server re-establishes the track for the obstacle that does not match the obstacle historical track information. The obstacle historical track information has a corresponding data structure. It should be noted that different types of detectors can be divided into main detectors and auxiliary detectors. Specifically, if the obstacle information that does not successfully match the obstacle historical track information belongs to the obstacle type corresponding to the main detector, the server establishes a new obstacle track information according to the obstacle information that does not successfully match the obstacle historical track information. If the obstacle information that does not successfully match the obstacle historical track information belongs to the obstacle type corresponding to the auxiliary detector, and the obstacle information that does not successfully match the obstacle historical track information appears for multiple frames in a row (that is, it meets the preset matching frame number), the server establishes a new obstacle track information according to the obstacle information that does not successfully match the obstacle historical track information. Among them, the new obstacle track information has a unique track identifier (for example, track_id), and the new obstacle track information is added to the obstacle historical track information. When the server detects that the new obstacle track information is successfully matched again (that is, the preset publishing rule), the server performs track publishing processing on the new obstacle track information. This behavior is to avoid the situation where each detector misses detection.

[0054] In the embodiment of the present invention, obstacle matching is performed on obstacle information corresponding to different types of detectors and obstacle historical trajectory information respectively to obtain a first obstacle matching result and a second obstacle matching result, and obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, thereby improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0055] The above describes the multi-detector result fusion tracking method in the embodiment of the present invention. The following describes the multi-detector result fusion tracking device in the embodiment of the present invention. Figure 3 , an embodiment of a multi-detector result fusion tracking device in an embodiment of the present invention includes:

[0056] A first acquisition module 301 is used to acquire obstacle historical trajectory information, first obstacle information corresponding to the deep learning detector, and second obstacle information corresponding to the rule strategy detector;

[0057] A matching module 302 is used to perform obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information through a preset multi-target tracking association module to obtain a first obstacle matching result corresponding to the deep learning detector and a second obstacle matching result corresponding to the rule strategy detector;

[0058] The division module 303 is used to divide the obstacle matching results and update the obstacle trajectory state according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information.

[0059] In the embodiment of the present invention, obstacle matching is performed on obstacle information corresponding to different types of detectors and obstacle historical trajectory information respectively to obtain a first obstacle matching result and a second obstacle matching result, and obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, thereby improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0060] See also Figure 4 Another embodiment of the multi-detector result fusion tracking device in the embodiment of the present invention includes:

[0061] A first acquisition module 301 is used to acquire obstacle historical trajectory information, first obstacle information and second obstacle information, where the first obstacle information and the second obstacle information are from different types of detectors;

[0062] A matching module 302, configured to perform obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information to obtain a first obstacle matching result and a second obstacle matching result;

[0063] The division module 303 is used to divide the obstacle matching results and update the obstacle trajectory state according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information.

[0064] Optionally, the first acquisition module 301 may also be specifically used for:

[0065] Receiving a multi-detector result fusion tracking request, and performing an authority verification process on the multi-detector result fusion tracking request to obtain an authority verification result;

[0066] When the authority verification result is passed, the obstacle historical trajectory information, the first obstacle information and the second obstacle information are read from the preset database according to the multi-detector result fusion tracking request.

[0067] Optionally, the matching module 302 may further include:

[0068] an association unit 3021, configured to perform obstacle association processing on the first obstacle information, the second obstacle information and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data;

[0069] The matching unit 3022 is configured to perform obstacle matching processing on the first obstacle association data and the second obstacle association data respectively according to a preset matching algorithm to obtain a first obstacle matching result and a second obstacle matching result.

[0070] Optionally, the associating unit 3021 may also be specifically configured to:

[0071] Extracting features of the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle feature data, second obstacle feature data, and obstacle historical trajectory feature data;

[0072] Correlation analysis is performed on the first obstacle feature data and the second obstacle feature data respectively with the obstacle historical trajectory feature data to obtain first obstacle association data and second obstacle association data.

[0073] Optionally, the division module 303 may further include:

[0074] The dividing unit 3031 is configured to divide the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information into obstacle matching results by using a preset matching check model to obtain an obstacle division result, wherein the obstacle division result includes a first current frame obstacle that successfully matches only the first obstacle information, a second current frame obstacle that successfully matches only the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information;

[0075] The processing unit 3032 is used to perform confidence processing and obstacle trajectory status update according to the obstacle division result and the obstacle historical trajectory information.

[0076] Optionally, the processing unit 3032 may also be specifically configured to:

[0077] When the obstacle division result is the first current frame obstacle, updating the obstacle trajectory state according to the first current frame obstacle and the obstacle historical trajectory information, the obstacle trajectory state including obstacle position, obstacle size, obstacle speed, obstacle acceleration and obstacle type;

[0078] When the obstacle classification result is the second current frame obstacle, classify the second current frame obstacle into a low-confidence obstacle or a high-confidence obstacle according to a preset confidence threshold, and perform an obstacle trajectory state update operation on the obstacle historical trajectory information according to the low-confidence obstacle or the high-confidence obstacle;

[0079] When the obstacle division result is the third current frame obstacle, the confidence level corresponding to the third current frame obstacle is increased.

[0080] Optionally, the multi-detector result fusion tracking device further includes:

[0081] The second acquisition module 304 is used to acquire obstacle information that has not been successfully matched with the obstacle historical trajectory information;

[0082] The publishing module 305 is used to generate new obstacle trajectory information according to the obstacle information that has not been successfully matched with the historical obstacle trajectory information, and perform trajectory publishing processing on the new obstacle trajectory information according to a preset publishing rule.

[0083] In the embodiment of the present invention, obstacle matching is performed on obstacle information from different types of corresponding detectors and obstacle historical trajectory information respectively to obtain a first obstacle matching result and a second obstacle matching result, and obstacle matching results are divided and obstacle trajectory status is updated according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, thereby improving the accuracy and precision of obstacle recognition and obstacle trajectory construction, thereby improving the safety and reliability of unmanned driving.

[0084] above Figure 3 and Figure 4 The multi-detector result fusion tracking device in the embodiment of the present invention is described in detail from a modular perspective. The multi-detector result fusion tracking device in the embodiment of the present invention is described in detail from a hardware processing perspective.

[0085] Figure 5 1 is a schematic diagram of the structure of a multi-detector result fusion tracking device provided by an embodiment of the present invention. The multi-detector result fusion tracking device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more massive storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of computer program operations in the multi-detector result fusion tracking device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of computer program operations in the storage medium 530 on the multi-detector result fusion tracking device 500.

[0086] The multi-detector result fusion tracking device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 5 The structure of the multi-detector result fusion tracking device shown does not constitute a limitation of the multi-detector result fusion tracking device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0087] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-detector result fusion tracking method.

[0088] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0090] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-detector result fusion tracking method, characterized in that: The multi-detector result fusion tracking method comprises: Obtaining obstacle historical trajectory information, first obstacle information, and second obstacle information, where the first obstacle information and the second obstacle information are from different types of detectors; Obstacle matching is performed on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information by a preset multi-target tracking association module to obtain a first obstacle matching result and a second obstacle matching result, wherein the obstacle matching results corresponding to the different types of detectors have different weights; performing obstacle matching result division and obstacle trajectory status update according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information; The dividing of obstacle matching results and updating of obstacle track status according to the first obstacle matching result, the second obstacle matching result and the obstacle historical track information includes: Performing obstacle matching result division on the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information through a preset matching check model to obtain an obstacle division result, wherein the obstacle division result includes a first current frame obstacle that successfully matches only the first obstacle information, a second current frame obstacle that successfully matches only the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information; Confidence processing and obstacle trajectory status update are performed according to the obstacle division result and the obstacle historical trajectory information.

2. The multi-detector result fusion tracking method according to claim 1, characterized in that: The obtaining of obstacle historical trajectory information, first obstacle information and second obstacle information includes: Receiving a multi-detector result fusion tracking request, and performing an authority verification process on the multi-detector result fusion tracking request to obtain an authority verification result; When the authority verification result is passed, the obstacle historical trajectory information, the first obstacle information and the second obstacle information are read from the preset database according to the multi-detector result fusion tracking request.

3. The multi-detector result fusion tracking method according to claim 1, characterized in that: in, The preset multi-target tracking association module includes a preset association model and a preset matching algorithm. Obstacle matching is performed on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information by the preset multi-target tracking association module to obtain a first obstacle matching result and a second obstacle matching result, including: performing obstacle association processing on the first obstacle information, the second obstacle information and the obstacle historical trajectory information through a preset association model to obtain first obstacle association data and second obstacle association data; Obstacle matching processing is performed on the first obstacle association data and the second obstacle association data respectively according to a preset matching algorithm to obtain a first obstacle matching result and a second obstacle matching result.

4. The multi-detector result fusion tracking method according to claim 3, characterized in that: The performing obstacle association processing on the first obstacle information, the second obstacle information and the obstacle historical trajectory information by using a preset association model to obtain first obstacle association data and second obstacle association data includes: Extracting features of the first obstacle information, the second obstacle information, and the obstacle historical trajectory information through a preset association model to obtain first obstacle feature data, second obstacle feature data, and obstacle historical trajectory feature data; Correlation analysis is performed on the first obstacle feature data and the second obstacle feature data respectively with the obstacle historical trajectory feature data to obtain first obstacle association data and second obstacle association data.

5. The multi-detector result fusion tracking method according to claim 1, characterized in that: The performing confidence processing and obstacle trajectory status updating according to the obstacle division result and the obstacle historical trajectory information includes: When the obstacle division result is the first current frame obstacle, updating the obstacle trajectory state according to the first current frame obstacle and the obstacle historical trajectory information, the obstacle trajectory state including obstacle position, obstacle size, obstacle speed, obstacle acceleration and obstacle type; When the obstacle classification result is the second current frame obstacle, classify the second current frame obstacle into a low-confidence obstacle or a high-confidence obstacle according to a preset confidence threshold, and perform an obstacle trajectory state update operation on the obstacle historical trajectory information according to the low-confidence obstacle or the high-confidence obstacle; When the obstacle division result is the third current frame obstacle, the confidence level corresponding to the third current frame obstacle is increased.

6. The multi-detector result fusion tracking method according to any one of claims 1 to 5, characterized in that: After dividing the obstacle matching results and updating the obstacle trajectory state according to the first obstacle matching result, the second obstacle matching result and the obstacle historical trajectory information, the multi-detector result fusion tracking method further includes: Obtaining obstacle information that has not been successfully matched with the obstacle historical trajectory information; New obstacle trajectory information is generated according to obstacle information that has not been successfully matched with the obstacle historical trajectory information, and trajectory publishing processing is performed on the new obstacle trajectory information according to a preset publishing rule.

7. A multi-detector result fusion tracking device, characterized in that: The multi-detector result fusion tracking device comprises: A first acquisition module, used to acquire obstacle historical trajectory information, first obstacle information and second obstacle information, where the first obstacle information and the second obstacle information are derived from different types of detectors; a matching module, configured to perform obstacle matching on the first obstacle information and the second obstacle information respectively with the obstacle historical trajectory information through a preset multi-target tracking association module to obtain a first obstacle matching result and a second obstacle matching result, wherein the obstacle matching results corresponding to different types of detectors have different weights; A division module, configured to divide the obstacle matching results and update the obstacle track state according to the first obstacle matching result, the second obstacle matching result and the obstacle historical track information; The dividing of obstacle matching results and updating of obstacle track status according to the first obstacle matching result, the second obstacle matching result and the obstacle historical track information includes: Performing obstacle matching result division on the first obstacle matching result, the second obstacle matching result, and the obstacle historical trajectory information through a preset matching check model to obtain an obstacle division result, wherein the obstacle division result includes a first current frame obstacle that successfully matches only the first obstacle information, a second current frame obstacle that successfully matches only the second obstacle information, and a third current frame obstacle that successfully matches both types of obstacle information; Confidence processing and obstacle trajectory status update are performed according to the obstacle division result and the obstacle historical trajectory information.

8. A multi-detector result fusion tracking device, characterized in that: The multi-detector result fusion tracking device comprises: a memory and at least one processor, wherein a computer program is stored in the memory; The at least one processor calls the computer program in the memory to enable the multi-detector result fusion tracking device to perform the multi-detector result fusion tracking method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-detector result fusion tracking method as described in any one of claims 1 to 6 is implemented.

Images