Target fusion method and device, electronic equipment and storage medium

Abstract

This application discloses a target fusion method, apparatus, electronic device, and storage medium. The method includes establishing a correlation relationship based on the target detection results output by the Rule-Based algorithm in the LiDAR target detection algorithm and the AI ​​algorithm in the deep learning model, respectively; performing correlation processing on the target detection results according to the correlation relationship; and using the result of the correlation processing as the final target fusion result. This application reduces the occurrence of jump situations and improves the robustness of the target fusion result.

Description

Technical Field

[0001] This application relates to the fields of autonomous driving and target fusion technology, and in particular to a target fusion method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of autonomous driving, 3D target detection based on LiDAR is of paramount importance. LiDAR 3D target detection technology can be divided into rule-based algorithms based on traditional algorithms and target detection methods based on deep learning model AI algorithms.

[0003] To ensure the stability of autonomous driving systems, most existing solutions combine traditional algorithms and deep learning methods to jointly detect and fuse the output results. As the vehicle moves, the shape of the target scanned by LiDAR constantly changes, which leads to instability in target detection by both traditional and deep learning algorithms. The target in the fused output may exhibit abrupt changes, posing difficulties for downstream nodes in the autonomous driving system and potentially affecting MPI (Mean Intervention Mileage), motion perception, and visual presentation. Summary of the Invention

[0004] This application provides a target fusion method, apparatus, electronic device, and storage medium to optimize target fusion results and reduce jumps.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a target fusion method, wherein the method includes:

[0007] Establish a correlation based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm;

[0008] Based on the aforementioned correlation, the target detection results are subjected to correlation processing;

[0009] The result of the correlation processing is used as the final target fusion result.

[0010] In some embodiments, the association includes a target speed association, a target location association, and a target ID association; the method further includes:

[0011] Based on any one or more of the following associations—the target velocity association, the target position association, and the target ID association—the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are correlated.

[0012] In some embodiments, establishing the association includes:

[0013] Based on the targets detected by the Rule-Based algorithm and the targets detected by the deep learning model AI algorithm, a first map table and a second map table are obtained;

[0014] Based on the unique target tracking identifier track_id in the first map table and the second map table, target association relationships are established to obtain the associate_map association list. There is a many-to-many association between each target tracking object track_obj in the association list. The target detection result output_id of each target is recorded, and a mapping relationship of output_obj_id = track_id is established. N is a natural number.

[0015] In some embodiments, the association processing of the target detection results based on the association relationship includes:

[0016] When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in a 1:1 manner, determine whether the current deep learning model AI algorithm has the output_id of a historical frame.

[0017] If such a frame exists, output the output_id of the historical frame of the AI ​​algorithm; otherwise, find the output_id of the associated historical frame of the Rule-Based algorithm based on the association relationship.

[0018] If the detection result of the deep learning model AI algorithm is not detected in the current frame, then query the number of detection results of the deep learning model AI algorithm in the association list of the detection results of the Rule-Based algorithm in the previous frame;

[0019] Based on the number of detection results in the deep learning model AI algorithm, determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm.

[0020] In some embodiments, determining whether there is an ID in the historical detection result association table of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm includes: determining whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm.

[0021] If so, determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that already has the IDs of the currently matching deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame;

[0022] If not, add the ID of the currently matching deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that do not currently match the ID of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame.

[0023] If the remaining ID does not exist in the target list of the deep learning model AI algorithm in the current frame, then delete the ID of the deep learning model AI algorithm in the second map table and output the historical attribute information of the current deep learning model AI algorithm.

[0024] If the remaining ID exists in the target list of the deep learning model AI algorithm in the current frame, then delete the existing AI-id from the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm;

[0025] After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

[0026] In some embodiments, the association processing of the target detection results based on the association relationship includes:

[0027] When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and a 1:N correlation,

[0028] Determine whether there is an output_id in the output result of the deep learning model AI algorithm. If there is, output the output_id of the deep learning model AI algorithm. If not, output the track_id of the deep learning model AI algorithm in the current frame.

[0029] By searching the association list of the Rule-Based algorithm, the detection results of deep learning model AI algorithms that do not have output_id in the current frame are output, and the undetected deep learning model AI algorithms that may appear in each mapping relationship are updated.

[0030] If the deep learning model AI algorithm is not detected in the current frame, then query the number of detection results of the deep learning model AI algorithm in the association list of the Rule-Based algorithm in the previous frame;

[0031] Based on the number of detection results in the deep learning model AI algorithm, determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm.

[0032] In some embodiments, determining whether there is an ID in the historical detection result association table of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm includes...

[0033] Determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that matches the current deep learning model AI algorithm;

[0034] If so, determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that already has the IDs of the currently matching deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame;

[0035] If not, add the ID of the currently matching deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that do not currently match the ID of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame.

[0036] If the remaining ID does not exist in the target list of the deep learning model AI algorithm in the current frame, then delete the ID of the deep learning model AI algorithm in the second map table and output the historical attribute information of the current deep learning model AI algorithm.

[0037] If the remaining ID exists in the target list of the deep learning model AI algorithm in the current frame, then delete the existing AI-id from the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm;

[0038] After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

[0039] In some embodiments, the association processing of the target detection results based on the association relationship includes:

[0040] When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and an N:1 correlation...

[0041] If the AI ​​algorithm fails, check if there is an output_id in the detection result of the deep learning model AI algorithm of the current frame obtained by the LiDAR scan. If there is, directly output the output_id of the deep learning model AI algorithm.

[0042] If not, check which Rule-Based algorithm and deep learning model AI algorithm in the previous frame have a mapping relationship, and obtain the output_id corresponding to the Rule-Based algorithm according to the mapping relationship;

[0043] If the detection results of the Rule-Based algorithm do not contain historical information, the tracked-id of the deep learning model AI algorithm will be output directly.

[0044] After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

[0045] In some embodiments, the method further includes:

[0046] When performing correlation processing on the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm respectively, if there is no intersection between the Rule-Based algorithm and the deep learning model AI algorithm in the current frame obtained by LiDAR scanning, the following operation is performed:

[0047] Based on the detection results of the Rule-Based algorithm, examine the correlation with the previous frame;

[0048] If the current frame does not match the Rule-Based algorithm in the detection results of the deep learning model AI algorithm, then query the previous frame to see if there is a Rule-Based algorithm that matches the current deep learning model AI algorithm.

[0049] In some embodiments, examining the correlation of the detection results from the Rule-Based algorithm with the previous frame includes:

[0050] If the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are associated in a 1:1 ratio in the previous frame, then query whether the output of the deep learning model AI exists in the current frame.

[0051] If it exists, set the output of the Rule-Based algorithm in the current frame to invisible and update the association relationship;

[0052] If it does not exist, the Rule-Based algorithm information of the previous frame is output directly. If the Rule-Based algorithm of the previous frame does not exist either, the tracking information of the Rule-Based algorithm of the current frame is output.

[0053] In some embodiments, when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are associated in the previous frame with a 1:N relationship, the system queries whether there is an associated output result of the AI ​​algorithm in the current frame.

[0054] If it exists, then set the Rule-Based algorithm for the current frame to invisible and update the association.

[0055] If it does not exist, output the information of the Rule-Based algorithm of the previous frame. If the Rule-Based algorithm does not exist in the previous frame either, output the tracking information of the Rule-Based algorithm of the current frame.

[0056] In some embodiments, if the detection result of the deep learning model AI algorithm does not match the Rule-Based algorithm in the current frame, the query to see if there is a Rule-Based algorithm matching the current deep learning model AI algorithm in the previous frame includes:

[0057] If the query in the previous frame contains a Rule-Based algorithm that matches the current deep learning model's AI algorithm, then the output of the current AI is set to the output of the Rule-Based algorithm in the previous frame, and the association is updated.

[0058] If no Rule-Based algorithm matches the current deep learning model AI algorithm in the previous frame, the tracking information of the current AI is output.

[0059] In some embodiments, the association processing of the target detection results based on the association relationship includes:

[0060] According to the aforementioned relationship, if the target IDs in the deep learning model AI algorithms are duplicated, the target ID in the target detection result of any one of the deep learning model AI algorithms will be displayed as the current output_id, and the target IDs of the remaining deep learning model AI algorithms will be displayed as track_id.

[0061] According to the aforementioned association, if the target ID in the Rule-Based algorithm is duplicated, the target ID in the target detection result of any Rule-Based algorithm will be displayed as the current output_id, and the target IDs of the remaining Rule-Based algorithms will be invisible.

[0062] According to the aforementioned relationship, if the IDs of the Rule-Based algorithm and the deep learning model AI algorithm are duplicated, the target ID in the target detection result of any one of the deep learning model AI algorithms will be the current output_id, the target ID of the remaining deep learning model AI algorithms will be track_id, and the target ID of the Rule-Based algorithm will be invisible.

[0063] Secondly, embodiments of this application also provide a target fusion apparatus, wherein the apparatus includes:

[0064] The association establishment module is used to establish associations based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm.

[0065] The association processing module is used to perform association processing on the target detection results according to the association relationship;

[0066] The fusion module is used to take the result of the correlation processing as the final target fusion result.

[0067] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0068] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0069] The at least one technical solution adopted in this application can achieve the following beneficial effects: A correlation is established by using the target detection results output by the Rule-Based algorithm and the AI ​​algorithm in the LiDAR target detection algorithm, respectively; the target detection results are then processed according to the correlation; and the result of the correlation processing is used as the final target fusion result. This application optimizes the target fusion result, making the result more stable and reducing jumps. Attached Figure Description

[0070] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0071] Figure 1 This is a schematic diagram of the target fusion method in the embodiments of this application;

[0072] Figure 2 This is a schematic diagram of the target fusion device structure in the embodiments of this application;

[0073] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] In related technologies, lidar target detection frameworks employ both traditional and deep learning algorithms to process point cloud data in parallel for target detection and tracking, before jointly entering a target fusion module. To output a robust fused target, this application proposes a lidar-based target fusion scheme that optionally acquires a baseline target (target object, including point cloud information, 3DBOX information, velocity information, and corner information) and ensures that the final target does not exhibit any abrupt changes.

[0076] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0077] This application provides a target fusion method, such as... Figure 1The diagram shows a schematic flow chart of a target fusion method in an embodiment of this application. The target fusion method includes at least the following steps S110 to S130:

[0078] Step S110: Establish a correlation between the target detection results output by the Rule-Based algorithm of the LiDAR target detection algorithm and the AI ​​algorithm of the deep learning model.

[0079] LiDAR is typically used in vehicles, meaning it's installed in a car with its calibration parameters adjusted to acquire real-time lane information. The vehicle also features onboard devices like the Xavier in-vehicle unit, capable of running deep learning models (AI algorithms) and traditional algorithms (RB, rule-based algorithms). It's important to note that the Xavier in-vehicle unit is merely an example and not intended to limit the scope of protection of this application.

[0080] The 3D target detection algorithm for LiDAR includes both rule-based algorithms and deep learning model AI algorithms. It should be noted that the rule-based algorithm refers to the LiDAR detection algorithm using traditional detection methods, while the deep learning model AI algorithm refers to the LiDAR detection algorithm using a novel detection method based on machine learning models. In the embodiments of this application, no specific limitation is made on the AI ​​algorithm or the rule-based algorithm. Those skilled in the art can choose according to the actual application scenario.

[0081] Establish a correlation between the target detection results output by the two algorithms. This correlation is related to the target object. For example, if the target object output by the two detection algorithms has the same speed, or the target object has the same target ID, etc., these are all cases where a relationship can be established.

[0082] Step S120: Based on the association relationship, perform association processing on the target detection results.

[0083] Based on the established association relationship, the target detection results are then processed and judged. It is possible that the target disappears from the detection results and then reappears after a period of time. In this case, it is necessary to ensure that the target IDs in the final target fusion output are consistent. This can solve the possible jump problem. Similarly, the target speed and target position are solved in the same way.

[0084] Step S130: The result of the association processing is used as the final target fusion result.

[0085] By establishing a relationship between the targets output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model, and then performing association processing on the targets, duplicate IDs of the targets can be eliminated.

[0086] By using the above method, optimizing the target fusion result can select the baseline target (referring to the point cloud target whose target fusion result meets the requirements) and ensure that the final target will not change, so that the fused target output in the final detection result is robust.

[0087] Unlike the rule-based algorithm and AI algorithm based on deep learning models that are prone to skipping when jointly detecting and fusing output results, this application establishes a correlation between the targets in the two detection algorithms and removes duplicates based on the correlation, thereby ensuring the accuracy of the target fusion result while reducing skipping.

[0088] In one embodiment of this application, the association relationship includes the target speed association relationship, the target location association relationship, and the target ID association relationship. The method further includes: performing association processing on the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm respectively, based on any one or more of the target speed association relationship, the target location association relationship, and the target ID association relationship.

[0089] Based on various correlations, the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm can be correlated. For example, based on positional correlation, target A detected by the Rule-Based algorithm can be correlated with target A' detected by the deep learning model AI algorithm, and target A and target A' can be considered to belong to the same point cloud target. Similarly, correlations can be based on velocity or target object correlations.

[0090] Furthermore, by establishing an association table and searching the table for the detection results of the Rule-Based algorithm and the deep learning model AI algorithm during each target fusion, potential targets can be identified. It's understood that a deep learning model AI algorithm's detection result can contain multiple Rule-Based algorithm detection results. The Rule-Based algorithm can be used to find the deep learning model AI algorithm and establish a matching relationship. Preferably, the deep learning model AI algorithm is prioritized during matching; if it's not found (potentially due to false positives or false negatives), the system checks for the presence of a Rule-Based algorithm. If a Rule-Based algorithm is found, its detection result is used.

[0091] In one embodiment of this application, establishing the association includes: obtaining a first map table and a second map table based on the targets detected by the Rule-Based algorithm and the targets detected by the deep learning model AI algorithm; establishing an N:N association relationship for targets based on the unique target tracking identifier track_id in the first map table and the second map table to obtain an associate_map association list, wherein there is a many-to-many association between each target tracking object track_obj in the association list, and by recording the target detection result output_id of each target, a mapping relationship of output_obj_id = track_id is established, where N is a natural number.

[0092] Among them, associate_map: is the ID of the corresponding RB or AI associated with it.

[0093] track_id: The individual tracking ID.

[0094] output_id: The ID of the final output, which needs to be recorded and saved.

[0095] output_obj_id: output_id, the target from which the output comes.

[0096] Rule-Based Algorithm: RB Algorithm.

[0097] Deep learning models and AI algorithms: AI algorithms.

[0098] Establishing a relationship specifically includes:

[0099] First, a mapping table (map) is established for the targets detected by the Rule-Based algorithm and the targets detected by the deep learning model AI algorithm, respectively, which records the key value and value of each target.

[0100] Then, the targets detected by the Rule-Based method and the targets detected by the deep learning model AI algorithm are associated with each other, using track_id as a unique identifier and N representing the number of targets.

[0101] Next, create the association list `associate_map`. <int,vector <int>> where the key is an int and the value is a vector. <int>Furthermore, each track_obj (target tracking object) has a many-to-many relationship.

[0102] Finally, record the output_id for each target and establish a mapping relationship of output_obj_id = track_id. Each target will be identified by its own ID during tracking; if there is an overlap, that ID will be output as the output_id.

[0103] In one embodiment of this application, the association processing of the target detection results based on the association relationship includes: when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in a 1:1 ratio, determining whether the current deep learning model AI algorithm has a historical frame output_id; if so, outputting the historical frame output_id; otherwise, searching for the output_id of the associated historical frame of the Rule-Based algorithm based on the association relationship; if the deep learning model AI algorithm is not detected in the current frame, querying the number of detection results of the deep learning model AI algorithm in the association list of the Rule-Based algorithm in the previous frame; and determining whether there is an ID of the currently matching deep learning model AI algorithm in the historical association table of the detection results of the deep learning model AI algorithm based on the number of detection results in the deep learning model AI algorithm.

[0104] The correlation processing of the target detection results specifically includes the following situations:

[0105] When the Rule-Based algorithm and the deep learning model AI algorithm intersect in the current frame, the output of the deep learning model AI algorithm takes precedence. It is then determined whether the current deep learning model AI algorithm has the output-id of a historical frame. If it does, the output-id of the historical frame is output; otherwise, the output-id of the historical frame of the associated Rule-Based algorithm is searched.

[0106] It should be noted that the current frame or historical frame mentioned above refers to the result of one frame in each scan by the LiDAR.

[0107] In one embodiment of this application, determining whether there is an ID matching the current deep learning model AI algorithm in the association table of historical detection results of the deep learning model AI algorithm includes: determining whether there is an ID matching the current deep learning model AI algorithm in the historical association table of detection results of the deep learning model AI algorithm; if so, determining whether the remaining IDs among the IDs matching the current deep learning model AI algorithm in the historical association table of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame; if not, adding the ID matching the current deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determining whether there is a current deep learning model AI algorithm in the historical association table of the deep learning model AI algorithm. The remaining IDs in the IDs are checked against the target list of the deep learning model AI algorithm in the current frame. If the remaining IDs do not exist in the target list of the deep learning model AI algorithm in the current frame, the IDs of the deep learning model AI algorithm in the second map table are deleted, and the historical attribute information of the current deep learning model AI algorithm is output. If the remaining IDs exist in the target list of the deep learning model AI algorithm in the current frame, the existing AI-ids in the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm are deleted. After the above steps, the associated list of the deep learning model AI algorithm and the Rule-Based algorithm, the associated list of associate_map, output_id, output_obj_id, and track_id are updated.

[0108] In practical implementation, if there are undetected deep learning model AI algorithms based on the above judgment results, the following further processing is required:

[0109] First, check the association list to see if there is one or multiple AI algorithms. Then, based on the query results, if the ID of the currently matching AI exists in the historical association table, check if the remaining IDs exist in the obj list of the AI ​​in the current frame. If they exist, delete the existing AI-id from the map table of the AI ​​associated with RB. If they do not exist, output the historical attribute information of these AIs; if no historical frame exists, directly delete the AI-id from the map table.

[0110] Additionally, if the historical association table does not contain the ID of the currently matching AI, the current AI-id is added to the map table of the AI ​​associated with the RB, and the remaining IDs are checked for existence in the obj list of the AI ​​in the current frame. If the ID exists, it is deleted from the map table of the AI ​​associated with the RB. If it does not exist, the historical attribute information of these AIs is output; if the historical frame does not exist, the AI-id is directly deleted from the map table. Furthermore, after the above steps, the associated_map, output_id, output_obj_id, and track_id corresponding to the AI ​​and RB are updated.

[0111] It should be noted that historical frames include the previous frame, the frame before that, etc., excluding the current frame. Historical attribute information includes the target velocity, target position, target type, etc., corresponding to the historical frame, which are not specifically limited in the embodiments of this application.

[0112] In one embodiment of this application, the association processing of the target detection results based on the association relationship includes: when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in a 1:N manner, determining whether the output result of the deep learning model AI algorithm contains an output_id; if so, outputting the output_id of the deep learning model AI algorithm; otherwise, outputting the track_id of the deep learning model AI algorithm in the current frame; by searching the association list of the Rule-Based algorithm, outputting the deep learning model AI algorithms that do not contain an output_id in the current frame, and updating any undetected deep learning model AI algorithms that may appear in each mapping relationship; if the deep learning model AI algorithm is not detected in the current frame, querying the number of detection results of the deep learning model AI algorithm in the association list of the Rule-Based algorithm in the previous frame; and determining whether there is an ID of the currently matching deep learning model AI algorithm in the historical association table of the detection results of the deep learning model AI algorithm based on the number of detection results in the deep learning model AI algorithm.

[0113] Check the RB association list and output the AIs that are not present in the current frame. Simultaneously, update the information for AIs that may not have been detected in each mapping relationship, specifically including:

[0114] Query the AI ​​in the associated list of the previous frame RB to determine whether it is one AI or multiple AIs.

[0115] If the historical association table contains the ID of the currently matching AI, check if the remaining IDs exist in the obj list of the AI ​​in the current frame. If they exist, delete the existing AI-id from the map table of the AI ​​associated with RB. If they do not exist, output the historical attribute information of these AIs; if the historical frame does not exist, directly delete the AI-id from the map table.

[0116] Simultaneously, if the historical association table does not contain the ID of the currently matching AI, the current AI-id is added to the map table of the AI ​​associated with RB, and the remaining IDs are checked for existence in the obj list of the AI ​​in the current frame. If the result shows that the AI-id exists, it is deleted from the map table of the AI ​​associated with RB. If it does not exist, the historical attribute information of these AIs is output; if the historical frame does not exist, the AI-id is directly deleted from the map table.

[0117] After the above steps, update the associated_map, output_id, output_obj_id, and track_id corresponding to AI and RB.

[0118] In one embodiment of this application, determining whether there is an ID matching the current deep learning model AI algorithm in the association table of historical detection results of the deep learning model AI algorithm includes determining whether there is an ID matching the current deep learning model AI algorithm in the historical association table of detection results of the deep learning model AI algorithm; if so, determining whether the remaining IDs among the IDs matching the current deep learning model AI algorithm in the historical association table of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame; if not, adding the ID matching the current deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determining whether there is an ID matching the current deep learning model AI algorithm in the historical association table of the deep learning model AI algorithm. If the remaining IDs in the ID table do not exist in the target list of the deep learning model AI algorithm in the current frame, then delete the IDs of the deep learning model AI algorithm in the second map table and output the historical attribute information of the current deep learning model AI algorithm; if the remaining IDs exist in the target list of the deep learning model AI algorithm in the current frame, then delete the existing AI-ids from the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm; after the above steps, update the associated list of the deep learning model AI algorithm and the Rule-Based algorithm, the associated list of ...

[0119] In practice, if the associated_map table contains the ID of the currently matching deep learning model AI algorithm, it checks whether the remaining IDs exist in the target object list (obj list) of the current frame's deep learning model AI algorithm. If they exist, the IDs of the existing AIs are deleted from the map mapping table associated with the Rule-Based algorithm deep learning model AI algorithm. If the historical frames do not exist, the historical attribute information of these deep learning model AIs will be output.

[0120] Furthermore, if the historical association table does not contain the ID of the current matching deep learning model AI algorithm, then the current AI-id needs to be added to the map table of the AI ​​associated with the Rule-Based algorithm, and it needs to be determined whether the remaining ID exists in the obj list of the current frame deep learning model AI algorithm.

[0121] Then, if the historical frame exists, delete the ID of the existing AI from the map table of the deep learning model AI algorithm associated with the Rule-Based algorithm. If the historical frame does not exist, output the historical attribute information of these deep learning model AI algorithms.

[0122] Next, update the associated_map list, output_id, output_obj_id of the target detection result, and track_id of the target tracking corresponding to the deep learning model AI algorithm and the Rule-Based algorithm.

[0123] In one embodiment of this application, the association processing of the target detection results based on the association relationship includes: when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in an N:1 ratio, after determining that the AI ​​algorithm has failed, determining whether there is an output_id in the detection results of the deep learning model AI algorithm in the current frame obtained by the LiDAR scan; if there is, directly outputting the output_id of the deep learning model AI algorithm; if not, checking which Rule-Based algorithm and deep learning model AI algorithm have a mapping relationship in the previous frame, and obtaining the output_id corresponding to the Rule-Based algorithm according to the mapping relationship; if there is no historical information in the detection results of the Rule-Based algorithm, directly outputting the tracked-id of the deep learning model AI algorithm; after the above steps, updating the associated_map association list, output_id, output_obj_id from which the target and track_id of the deep learning model AI algorithm and the Rule-Based algorithm are output.

[0124] When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are respectively associated in an N:1 ratio, i.e., N:1 association (RB:AI);

[0125] When the deep learning model AI algorithm is detected to have disappeared or become unresponsive, it is necessary to determine whether the deep learning model AI algorithm in the current frame has an output_id. If it does, the output_id of the deep learning model AI algorithm is directly output; otherwise, it is necessary to check which Rule-Based algorithm in the previous frame is mapped to the deep learning model AI algorithm (output_id) and retrieve the output_id corresponding to the mapped Rule-Based algorithm.

[0126] Furthermore, if the Rule-Based algorithm also lacks historical information, it directly outputs the unique target tracking identifier tracked-id of the deep learning model AI algorithm. In the current frame, the Rule-Based algorithm takes the most recent (previous frame or the frame from which target matching and association began) and maps and associates it with the deep learning model AI algorithm. Finally, it updates the associated_map list, output_id target detection result, output_obj_id target from which the output comes, and track_id unique target tracking identifier of the deep learning model AI algorithm and the Rule-Based algorithm.

[0127] In one embodiment of this application, the method further includes: when performing correlation processing on the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm respectively, if there is no intersection between the Rule-Based algorithm and the deep learning model AI algorithm in the current frame obtained by LiDAR scanning, the following operations are performed: for the detection results of the Rule-Based algorithm, check the correlation situation of the previous frame; for the detection results of the deep learning model AI algorithm, if the current frame does not match the Rule-Based algorithm, query whether there is a Rule-Based algorithm in the previous frame that matches the current deep learning model AI algorithm.

[0128] Besides the associated N:N, 1:N, and N:1 cases, there may also be situations where the outputs of the two algorithms do not overlap. When the Rule-Based algorithm and the deep learning model AI algorithm do not overlap, the following processing is performed:

[0129] a. For the Rule-Based algorithm, examine the correlation of the previous frame.

[0130] b. For deep learning model AI algorithms, if no Rule-Based algorithm is matched in the current frame, query the previous frame to see if there is a Rule-Based algorithm that matches the current deep learning model AI algorithm.

[0131] In summary, the results obtained can optimize the target fusion results when there is no overlap.

[0132] In one embodiment of this application, the step of checking the correlation of the detection results of the Rule-Based algorithm with the previous frame includes: when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are correlated in a 1:1 ratio in the previous frame, then query whether the output of the deep learning model AI algorithm exists in the current frame; if it exists, the Rule-Based algorithm in the current frame is set to invisible and the correlation is updated; if it does not exist, the Rule-Based algorithm information of the previous frame is directly output; if the Rule-Based algorithm in the previous frame also does not exist, the tracking information of the Rule-Based algorithm in the current frame is output.

[0133] a. For the Rule-Based algorithm, examine the correlation of the previous frame.

[0134] a1-1. If the association state of the previous frame is (1:1), then query whether the current frame contains the output of the deep learning model AI from the previous frame.

[0135] a1-2. If it exists, the Rule-Based algorithm for the current frame is set to invisible, and the association relationship is updated. At this time, the confidence of the deep learning model AI is relatively high.

[0136] a1-3. If it does not exist, output the information of the Rule-Based algorithm of the previous frame directly. If the Rule-Based algorithm of the previous frame does not exist either, output the tracking information of the Rule-Based algorithm of the current frame.

[0137] In one embodiment of this application, when the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have a 1:N association status in the previous frame, the system queries whether there is an associated output result of the deep learning model AI algorithm in the current frame. If there is, the Rule-Based algorithm in the current frame is made invisible, and the association relationship is updated. If there is no such output, the information of the Rule-Based algorithm in the previous frame is output. If the Rule-Based algorithm in the previous frame is also not present, the tracking information of the Rule-Based algorithm in the current frame is output.

[0138] a. For the Rule-Based algorithm, examine the correlation of the previous frame.

[0139] a1-4. If the association state of the previous frame is (1:N), then it is necessary to query whether the current frame has an associated deep learning model AI algorithm output.

[0140] a1-5. If it exists, make the Rule-Based algorithm invisible in the current frame and update the association relationship. If it does not exist, directly output the information of the Rule-Based algorithm in the previous frame. If the Rule-Based algorithm in the previous frame also does not exist, output the tracking information of the Rule-Based algorithm in the current frame.

[0141] In one embodiment of this application, the detection result of the deep learning model AI algorithm, if no Rule-Based algorithm is matched in the current frame, then querying whether there is a Rule-Based algorithm matching the current deep learning model AI algorithm in the previous frame, includes: if there is a Rule-Based algorithm matching the current deep learning model AI algorithm in the previous frame, then setting the output of the current AI to the output of the Rule-Based algorithm in the previous frame and updating the association relationship; if there is no Rule-Based algorithm matching the current deep learning model AI algorithm in the previous frame, then outputting the tracking information of the current AI.

[0142] b. For deep learning model AI algorithms, if no Rule-Based algorithm is matched in the current frame, query the previous frame to see if there is a Rule-Based algorithm that matches the current deep learning model AI algorithm.

[0143] b-1. If present, set the output of the current deep learning model's AI algorithm to the output of the previous frame's Rule-Based algorithm and update the association relationships.

[0144] b-2. If not, output the tracking information of the current deep learning model AI algorithm. Finally, based on the updated association relationships, update the associate_map, output_id, and track_id of each object, and update the association mapping table.

[0145] In one embodiment of this application, the association processing of the target detection results according to the association relationship includes: according to the association relationship, if the target IDs in the deep learning model AI algorithms are duplicated, then the target IDs in the target detection results of any one of the deep learning model AI algorithms are displayed as the current output_id, and the target IDs of the remaining deep learning model AI algorithms are displayed as track_id; according to the association relationship, if the target IDs in the rule-based algorithms are duplicated, then the target IDs in the target detection results of any one of the rule-based algorithms are displayed as the current output_id, and the target IDs of the remaining rule-based algorithms are not visible; according to the association relationship, if the IDs of the rule-based algorithms and the deep learning model AI algorithms are duplicated, then the target IDs in the target detection results of any one of the deep learning model AI algorithms are displayed as the current output_id, the target IDs of the remaining deep learning model AI algorithms are displayed as track_id, and the target IDs of the rule-based algorithms are not visible.

[0146] When handling duplicate IDs in the results, the following conditions are considered: Does the associated table for output_id have a value, and is the track-id relatively small?

[0147] If there is a duplicate ID between deep learning models / AI algorithms, use the current output-id for one of the deep learning models / AI algorithms, and use the track-id for the other AI targets.

[0148] If IDs are duplicated among Rule-Based algorithms, the current output-id will be displayed for one Rule-Based algorithm, while the target will remain invisible for other Rule-Based algorithms.

[0149] If there is a duplicate ID between a deep learning model AI algorithm and a rule-based algorithm, the deep learning model AI algorithm will use the current output-id, while the other deep learning model AI algorithms will use the track-id. The rule-based algorithm will not be visible.

[0150] This application embodiment also provides a target fusion device 200, such as Figure 2 As shown, a schematic diagram of the target fusion device in an embodiment of this application is provided. The target fusion device 200 includes at least: an association relationship establishment module 210, an association processing module 220, and a fusion module 230, wherein:

[0151] In one embodiment of this application, the association establishment module 210 is specifically used to: establish an association between the target detection results output by the Rule-Based algorithm of the LiDAR target detection algorithm and the AI ​​algorithm of the deep learning model.

[0152] LiDAR is typically used in vehicles, meaning it's installed in a car with properly calibrated radar parameters to acquire real-time lane information. The vehicle also features an onboard device similar to Xavier that can run deep learning models (AI algorithms) and traditional algorithms (RB, rule-based algorithms). It's important to note that the Xavier onboard device is merely an example and is not intended to limit the scope of protection of this application.

[0153] The 3D target detection algorithm for LiDAR includes both rule-based algorithms and deep learning model AI algorithms. It should be noted that the rule-based algorithm refers to the LiDAR detection algorithm using traditional detection methods, while the deep learning model AI algorithm refers to the LiDAR detection algorithm using a novel detection method based on machine learning models. In the embodiments of this application, no specific limitation is made on the AI ​​algorithm or the rule-based algorithm. Those skilled in the art can choose according to the actual application scenario.

[0154] Establish a correlation between the target detection results output by the two algorithms. This correlation is related to the target object. For example, if the target object output by the two detection algorithms has the same speed, or the target object has the same target ID, etc., these are all cases where a relationship can be established.

[0155] In one embodiment of this application, the association processing module 220 is specifically used to: perform association processing on the target detection results according to the association relationship.

[0156] Based on the established association, the target detection results are then further processed. It is possible that the Rule-Based algorithm does not detect the target disappearing in the detection results, but it will reappear after a period of time. In this case, it is necessary to ensure that the target IDs in the final target fusion output are consistent. This can solve the possible jump problem. Similarly, the speed and position of the target are solved in the same way.

[0157] In one embodiment of this application, the fusion module 230 is specifically used to: use the result of the association processing as the final target fusion result.

[0158] By establishing a relationship between the targets output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model, and then performing association processing on the targets, duplicate IDs of the targets can be eliminated.

[0159] It is understood that the above-mentioned target fusion device can realize each step of the target fusion method provided in the foregoing embodiments. The relevant explanations of the target fusion method are applicable to the target fusion device and will not be repeated here.

[0160] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0161] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0162] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0163] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming the target fusion device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0164] Establish a correlation based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm;

[0165] Based on the aforementioned correlation, the target detection results are subjected to correlation processing;

[0166] The result of the correlation processing is used as the final target fusion result.

[0167] The above is as stated in this application. Figure 1 The method executed by the target fusion device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0168] The electronic device can also perform Figure 1 The method for executing the target fusion device, and realizing the target fusion device in Figure 1 The functions of the embodiments shown are not described in detail here.

[0169] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the target fusion device in the illustrated embodiment is specifically used to perform:

[0170] Establish a correlation based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm;

[0171] Based on the aforementioned correlation, the target detection results are subjected to correlation processing;

[0172] The result of the correlation processing is used as the final target fusion result.

[0173] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0177] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0178] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0179] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.< / int> < / int>

Claims

1. A target fusion method, wherein, The method includes: Establish a correlation based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm; Based on the aforementioned correlation, the target detection results are subjected to correlation processing; The association processing of the target detection results based on the association relationship includes: When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in a 1:1 manner, determine whether the current deep learning model AI algorithm has the output_id of a historical frame. If such a frame exists, output the output_id of the historical frame of the AI ​​algorithm; otherwise, find the output_id of the associated historical frame of the Rule-Based algorithm based on the association relationship. If the detection result of the deep learning model AI algorithm is not detected in the current frame, then query the number of detection results of the deep learning model AI algorithm in the associated list of the detection results of the Rule-Based algorithm in the previous frame; Based on the number of detection results in the deep learning model AI algorithm, determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that matches the current detection result of the deep learning model AI algorithm; The result of the correlation processing is used as the final target fusion result.

2. The method of claim 1, wherein, The association relationships include target speed association relationships, target location association relationships, and target ID association relationships. The method further includes: Based on any one or more of the following associations—the target velocity association, the target position association, and the target ID association—the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are correlated.

3. The method of claim 2, wherein, The establishment of the association includes: Based on the targets detected by the Rule-Based algorithm and the targets detected by the deep learning model AI algorithm, a first map table and a second map table are obtained respectively. Based on the unique target tracking identifier track_id in the first map table and the second map table, target association relationships are established to obtain the associate_map association list. There is a many-to-many association between each target tracking object track_obj in the association list. The target detection result output_id of each target is recorded, and a mapping relationship of output_obj_id = track_id is established. N is a natural number.

4. The method of claim 1, wherein, The step of determining whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that matches the current deep learning model AI algorithm includes: Determine if the historical association table of the deep learning model AI algorithm contains an ID that matches the current deep learning model AI algorithm. If so, determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that already has the IDs of the currently matching deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame; If not, add the ID of the currently matching deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that do not currently match the ID of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame. If the remaining ID does not exist in the target list of the deep learning model AI algorithm in the current frame, then delete the ID of the deep learning model AI algorithm in the second map table and output the historical attribute information of the current deep learning model AI algorithm. If the remaining ID exists in the target list of the deep learning model AI algorithm in the current frame, then delete the existing AI-id from the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm; After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

5. The method of claim 3, wherein, The association processing of the target detection results based on the association relationship includes: When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and a 1:N correlation, Determine whether there is an output_id in the output result of the deep learning model AI algorithm. If there is, output the output_id of the deep learning model AI algorithm. If not, output the track_id of the deep learning model AI algorithm in the current frame. By searching the association list of the Rule-Based algorithm, the detection results of deep learning model AI algorithms that do not have output_id in the current frame are output, and the undetected deep learning model AI algorithms that may appear in each mapping relationship are updated. If the deep learning model AI algorithm is not detected in the current frame, then query the number of detection results of the deep learning model AI algorithm in the association list of the Rule-Based algorithm in the previous frame; Based on the number of detection results in the deep learning model AI algorithm, determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that currently matches the deep learning model AI algorithm.

6. The method of claim 4, wherein, The step of determining whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that matches the current deep learning model AI algorithm includes: Determine whether there is an ID in the historical association table of the deep learning model AI algorithm that matches the current deep learning model AI algorithm; If so, determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that already has the IDs of the currently matching deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame; If not, add the ID of the currently matching deep learning model AI algorithm to the first map table of the deep learning model AI algorithm associated with the Rule-Based algorithm, and then determine whether the remaining IDs in the historical association table of the deep learning model AI algorithm that do not currently match the ID of the deep learning model AI algorithm exist in the target list of the deep learning model AI algorithm in the current frame. If the remaining ID does not exist in the target list of the deep learning model AI algorithm in the current frame, then delete the ID of the deep learning model AI algorithm in the second map table and output the historical attribute information of the current deep learning model AI algorithm. If the remaining ID exists in the target list of the deep learning model AI algorithm in the current frame, then delete the existing AI-id from the second map table of the deep learning model AI algorithm associated with the Rule-Based algorithm; After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

7. The method of claim 3, wherein, The association processing of the target detection results based on the association relationship includes: When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and an N:1 correlation... Determine whether there is an output_id in the detection result of the deep learning model AI algorithm in the current frame. If there is, directly output the output_id of the deep learning model AI algorithm. If not, check which Rule-Based algorithm and deep learning model AI algorithm in the previous frame have a mapping relationship, and obtain the output_id corresponding to the Rule-Based algorithm according to the mapping relationship; If the detection results of the Rule-Based algorithm do not contain historical information, the tracked-id of the deep learning model AI algorithm will be output directly. After the above steps, update the associated_map list, output_id, output_obj_id, and track_id in the deep learning model AI algorithm and the rule-based algorithm.

8. The method of claim 1, wherein, The method further includes: When performing correlation processing on the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm respectively, if there is no intersection between the Rule-Based algorithm and the deep learning model AI algorithm in the current frame obtained by LiDAR scanning, the following operation is performed: Based on the detection results of the Rule-Based algorithm, examine the correlation with the previous frame; If the current frame does not match the Rule-Based algorithm in the detection results of the deep learning model AI algorithm, then query the previous frame to see if there is a Rule-Based algorithm that matches the current deep learning model AI algorithm.

9. The method of claim 8, wherein: The process of examining the detection results of the Rule-Based algorithm and checking the correlation with the previous frame includes: If the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are associated in a 1:1 ratio in the previous frame, then query whether the output of the deep learning model AI exists in the current frame. If it exists, set the output of the Rule-Based algorithm in the current frame to invisible and update the association relationship; If it does not exist, the Rule-Based algorithm information of the previous frame is output directly. If the Rule-Based algorithm of the previous frame does not exist either, the tracking information of the Rule-Based algorithm of the current frame is output.

10. The method of claim 8, wherein: If the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm are associated in the previous frame with a 1:N relationship, then query whether there is an associated output result of the deep learning model AI algorithm in the current frame. If it exists, then set the Rule-Based algorithm for the current frame to invisible and update the association. If it does not exist, output the information of the Rule-Based algorithm of the previous frame. If the Rule-Based algorithm does not exist in the previous frame either, output the tracking information of the Rule-Based algorithm of the current frame.

11. The method of claim 8, wherein, If the detection result of the deep learning model AI algorithm does not match the Rule-Based algorithm in the current frame, then the query is performed to see if there is a Rule-Based algorithm in the previous frame that matches the current deep learning model AI algorithm, including: If a Rule-Based algorithm matching the current deep learning model's AI algorithm is found in the previous frame, then the output of the current deep learning model's AI algorithm is set to the output of the Rule-Based algorithm in the previous frame, and the association is updated. If no Rule-Based algorithm matching the current deep learning model AI algorithm is found in the previous frame, the tracking information of the current deep learning model AI algorithm is output.

12. The method of claim 1, wherein, The association processing of the target detection results based on the association relationship includes: According to the aforementioned relationship, if the target IDs in the deep learning model AI algorithms are duplicated, the target ID in the target detection result of any one of the deep learning model AI algorithms will be displayed as the current output_id, and the target IDs of the remaining deep learning model AI algorithms will be displayed as track_id. According to the association, if the target ID in the Rule-Based algorithm is duplicated, the target ID in the target detection result of any Rule-Based algorithm will display the current output_id, and the target IDs of the remaining Rule-Based algorithms will be set to invisible. According to the aforementioned relationship, if the IDs of the Rule-Based algorithm and the deep learning model AI algorithm are duplicated, the target ID in the target detection result of any one of the deep learning model AI algorithms will be the current output_id, the target ID of the remaining deep learning model AI algorithms will be track_id, and the target ID of the Rule-Based algorithm will be set to invisible.

13. A target fusion device, wherein, The device includes: The association establishment module is used to establish associations based on the target detection results output by the Rule-Based algorithm and the AI ​​algorithm of the deep learning model in the LiDAR target detection algorithm. The association processing module is used to perform association processing on the target detection results according to the association relationship; The association processing of the target detection results based on the association relationship includes: When the target detection results output by the Rule-Based algorithm and the deep learning model AI algorithm have an intersection and are associated in a 1:1 manner, determine whether the current deep learning model AI algorithm has the output_id of a historical frame. If such a frame exists, output the output_id of the historical frame of the AI ​​algorithm; otherwise, find the output_id of the associated historical frame of the Rule-Based algorithm based on the association relationship. If the detection result of the deep learning model AI algorithm is not detected in the current frame, then query the number of detection results of the deep learning model AI algorithm in the associated list of the detection results of the Rule-Based algorithm in the previous frame; Based on the number of detection results in the deep learning model AI algorithm, determine whether there is an ID in the historical association table of the detection results of the deep learning model AI algorithm that matches the current detection result of the deep learning model AI algorithm; The fusion module is used to take the result of the correlation processing as the final target fusion result.

14. An electronic device comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 12.

15. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 12.

Images