Target association method and device, detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a target association method and device, a detection method and device, electronic equipment and a storage medium. The target association method comprises the steps that target detection frames of multiple sensors are acquired; determining a measurement index of a target detection frame according to the target detection frame of the plurality of sensors; a pre-trained target classification model is input in response to the measurement index of the target detection frame, a target association result output by the pre-trained target classification model is obtained, and the pre-trained target classification model is obtained through machine learning training by using multiple groups of sample data. Each group of data in the multiple groups of sample data comprises a measurement index of a target detection frame and a label of the target detection frame, and the label represents whether targets are associated or not. On one hand, the accuracy of multi-sensor fusion target association is improved, and on the other hand, the problems of poor user body feeling, multiple takeover and the like caused by mismatching are reduced.

Description

Technical Field

[0001] The present application relates to the technical field of multi-sensor target association, and in particular to a target association method, a detection method, a device, an electronic device, and a storage medium. Background Art

[0002] Modern cars have introduced many driving assistance functions, among which multi-sensor fusion algorithms are particularly critical, which aims to unify sensors from different sources into one goal. Especially for assisted driving functions such as vehicle road planning and lane keeping, multi-sensor fusion can integrate information from different sensors to provide vehicles with more comprehensive and accurate environmental perception, thereby ensuring that the vehicle can make correct decisions and actions during driving.

[0003] The current algorithms mainly use the traditional Euclidean distance to measure the target frame distance of multiple sensors. Some algorithms also add multi-dimensional information, but still use rules to calculate the comprehensive distance by using human experience to assign weights, which can only solve some fixed scenarios. Therefore, when facing complex scenarios such as mixed traffic of people and vehicles, it is easy to have misassociations. What's more serious is that some mismatches can cause the autonomous driving vehicle to brake suddenly, causing passengers to panic and feel unwell, and may even cause some potential safety hazards. Summary of the invention

[0004] The embodiments of the present application provide a target association method, a detection method, an apparatus, an electronic device, and a storage medium to improve the accuracy of multi-sensor fusion target association, thereby reducing problems such as physical sensation caused by mismatching and multiple takeovers.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a target association method, wherein the method comprises:

[0007] Get target detection frames from multiple sensors;

[0008] Determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors;

[0009] In response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and a label of the target detection box, and the label indicates whether the targets are associated.

[0010] In some embodiments, the sensor includes at least one of the following: a laser radar, a camera, and a millimeter wave radar, and obtaining target detection frames of multiple sensors includes:

[0011] The corner point coordinates, center point coordinates, speed information, category information and sensor attribute information of the 3D target box of each target detection box in multiple sensors are obtained respectively.

[0012] In some embodiments, after acquiring the target detection frames of multiple sensors, the method further includes:

[0013] Generate target trajectory and target detection tracking frame according to the associated target detection frame;

[0014] Align the timestamp of the target trajectory with the timestamp of the sensor, perform target association in the same time coordinate system, and obtain the historical target detection and tracking frame;

[0015] The Kalman filter is used to predict the spatial position of the target trajectory in the sensor to obtain a predicted target detection tracking frame;

[0016] According to the historical target detection and tracking frame and the predicted target detection and tracking frame, a fused trajectory target prediction frame is output.

[0017] In some embodiments, determining the metric of the target detection frame according to the target detection frames of the multiple sensors includes:

[0018] Calculate the Euclidean distance, intersection over union (IOU), speed distance, and category distance metrics between any two targets based on the target detection frames of the multiple sensors;

[0019] Setting a weight coefficient of the measurement indicator, and performing weighted summation on each measurement indicator according to the weight coefficient to obtain a comprehensive distance;

[0020] Calculate the optimal matching result according to the comprehensive distance to obtain the optimal matching result;

[0021] According to the optimal matching result, a label is set for each target pair.

[0022] In some embodiments, the pre-trained target classification model includes:

[0023] During the training phase of the model, the value of the loss function is calculated based on the prediction results of the current model in each iteration;

[0024] Calculate the gradient of the loss function with respect to the model parameters through the back-propagation algorithm;

[0025] Update the model parameters according to the gradient to minimize the value of the loss function.

[0026] In some embodiments, the inputting of the metric indicator of the target detection box into a pre-trained target classification model to obtain the target association result output by the pre-trained target classification model includes:

[0027] Loading the pre-trained target classification model;

[0028] In response to the metric of the target detection box, a pre-trained target classification model is input to predict and output a probability value of the result as a judgment of whether different target pairs belong to the same trajectory;

[0029] Input the probability value into the Hungarian algorithm to calculate the optimal match;

[0030] After completing the calculation of the optimal match, obtain the multi-sensor fusion trajectory target frame.

[0031] In a second aspect, an embodiment of the present application further provides a target detection method, wherein the target association method described in the first aspect is adopted.

[0032] In a third aspect, an embodiment of the present application further provides a target association device, wherein the device includes:

[0033] An acquisition module is used to obtain target detection frames of multiple sensors;

[0034] A determination module, configured to determine a measurement index of the target detection frame according to the target detection frames of the multiple sensors;

[0035] A response module is used to respond to the measurement index of the target detection frame input into a pre-trained target classification model to obtain a target association result output by the pre-trained target classification model, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection frame and the label of the target detection frame, and the label indicates whether the targets are associated.

[0036] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.

[0038] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: obtaining target detection frames of multiple sensors, and then determining the measurement index of the target detection frame according to the target detection frames of the multiple sensors. In response to the measurement index of the target detection frame, a pre-trained target classification model is input to obtain a target association result output by the pre-trained target classification model. The pre-trained target classification model is obtained through machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection frame and the label of the target detection frame, and the label indicates whether the targets are associated. Using the pre-trained target classification model, accurate classification is performed to reduce misjudgment and missed detection. The above method is used to improve the accuracy and efficiency of multi-target association while improving the performance of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 This is a schematic diagram of the overall implementation process of the target association method in the embodiment of the present application;

[0041] Figure 2 Schematic diagram of the process of the target association method in the embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the implementation principle of the target association method in the preferred embodiment of the present application;

[0043] Figure 4 This is a schematic diagram of the structure of the target association device in the embodiment of the present application;

[0044] Figure 5 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0046] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0047] like Figure 1As shown, it is a schematic diagram of the overall implementation process of the target association method in the embodiment of the present application, which mainly includes three steps, namely, collecting multi-target association samples, training the XGBoost model, and deploying the XGBoost model. Through these steps, accurate association of multiple targets can be achieved, and the accuracy and efficiency of multi-target tracking can be improved.

[0048] Specifically, in the step of collecting multi-target associated samples, the target detection and tracking results of multiple sensors are used as the original input to perform time alignment and one-step prediction of Kalman filtering on the fused trajectory. The Euclidean distance, IOU (intersection over union), speed, and category of the target box are calculated respectively, and pre-labeled using a manually designed rule algorithm. Finally, the correct associated samples are manually screened. It should be noted that for the results detected by each sensor, the respective sensor will track.

[0049] In the step of training the XGBoost model, the XGBoost model and loss function are designed. The training process includes data preprocessing, model parameter setting and initialization, calculation loss and gradient back propagation, and finally model prediction and model evaluation are performed to save the model weights with the best performance. The XGBoost model is a binary model, and the output is a value in the range of (0-1). It should be noted that the XGBoost model is only for example and is not used to limit the scope of protection in the embodiments of the present application.

[0050] In the step of deploying the XGBoost model, load the model parameters for initialization, input the calculated features for prediction, set the threshold, input the results into the Hungarian algorithm to calculate the optimal match, and finally obtain the fusion trajectory target frame of multiple sensors. The Hungarian algorithm can achieve global optimal matching.

[0051] The present application embodiment provides a target association method, such as Figure 2 As shown, a schematic flow chart of a target association method in an embodiment of the present application is provided, and the method at least includes the following steps S210 to S230:

[0052] Step S210: obtaining target detection frames of multiple sensors.

[0053] Obtain the target detection and tracking results sensed by each of the multiple sensors on the vehicle to obtain the target detection frames of the multiple sensors. It should be noted that the sensors installed on the vehicle include but are not limited to laser radar, camera, millimeter wave radar and other sensors, and the number is not specifically limited. Each sensor has its own target perception result and tracking result.

[0054] Step S220: determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors.

[0055] For the target detection frames obtained from multiple sensors, we can further calculate metrics such as Euclidean distance, IOU (Intersection over Union), speed distance, and category distance. More metrics are designed for multi-target association. By introducing multi-dimensional metrics such as Euclidean distance, IOU (Intersection over Union), speed, and category, we not only enrich the perspective of evaluation, but also significantly improve the comprehensiveness and accuracy of the measurement results. Multi-dimensional considerations can more accurately capture and analyze the complex relationships between targets, thereby providing more reliable decision support in practical applications.

[0056] Step S230, in response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and the label of the target detection box, and the label indicates whether the targets are associated.

[0057] In the training phase of the model, the pre-trained target classification model is obtained by machine learning training using multiple sets of sample data. For the training sample data, each set of data in the multiple sets of sample data includes: the metric of the target detection frame, i.e., each specific metric, and the label of the target detection frame, i.e., the label of whether the targets are associated.

[0058] After the metric index of the target detection frame is input into the pre-trained target classification model, the target association result output by the pre-trained target classification model is obtained. The multi-dimensional information fusion representation of the target is realized, the limitations of artificial rules are avoided, and complex nonlinear relationships can be processed, thereby exerting its unique advantages in target recognition and association analysis.

[0059] Through the above method, a pre-trained target classification model can be used to obtain a judgment result on whether the targets in the target detection frame are associated, which is convenient for more accurate capture and analysis of the complex relationship between the targets.

[0060] Through the above method, target detection frames of multiple sensors are obtained; and the measurement index of the target detection frame is determined according to the target detection frames of the multiple sensors. The measurement index of the target detection frame determined according to the target detection frames of multiple sensors enriches the evaluation perspective of whether the targets are associated, and also significantly improves the comprehensiveness and accuracy of the measurement results.

[0061] In one embodiment of the present application, the sensor includes at least one of the following: a laser radar, a camera, and a millimeter wave radar, and obtaining the target detection frames of multiple sensors includes: respectively obtaining the corner point coordinates, center point coordinates, speed information, category information, and sensor attribute information of the 3D target frame of each target detection frame in the multiple sensors.

[0062] The vehicle has been equipped with sensors such as on-board laser radar, camera, and millimeter-wave radar. By adjusting the parameters, relevant perception information of the surrounding environment can be obtained in real time. The corresponding detection algorithm can be used to obtain the observation value of the target frame, including:

[0063] 3D target box corner coordinates (X s , Y s , Z s )={(x1, y1, z1), (x2, y2, z2),..., (x n ,y n , z n )}; Center point coordinates Speed ​​V s =(v x , v y , v z ); category type; sensor attribute is the sensor id.

[0064] In one embodiment of the present application, after acquiring the target detection frames of multiple sensors, it also includes: generating a target trajectory and a target detection tracking frame based on the associated target detection frame; aligning the timestamp of the target trajectory with the timestamp of the sensor, performing target association in the same time coordinate system, and obtaining a historical target detection tracking frame; using Kalman filtering to predict the spatial position of the target trajectory in the sensor to obtain a predicted target detection tracking frame; and outputting a fused trajectory target prediction frame based on the historical target detection tracking frame and the predicted target detection tracking frame.

[0065] Obtain the results of time alignment processing and Kalman filter algorithm prediction, and then output the fusion trajectory target prediction frame based on the historical target detection and tracking frame and the predicted target detection and tracking frame.

[0066] Using time alignment, in the process of multi-target tracking and association, time consistency is crucial. The timestamp T of the target trajectory (i.e., the fusion result) is t The timestamp T of the sensor s , ensuring that target association is performed in the same time coordinate system.

[0067] Furthermore, the Kalman filter is used to predict the spatial position of the target trajectory in the sensor (the position of the current frame is determined based on the historical frames).

[0068] It can be understood that Kalman filtering can use the historical state of the target and the current observation. and Make a more accurate prediction of the target's future position. By continuously updating the difference between the prediction and the observation, the target trajectory estimate is gradually corrected to make it closer to the actual situation. This step provides an accurate spatial basis for subsequent distance calculation and association.

[0069] In one embodiment of the present application, determining the measurement indicators of the target detection frames according to the target detection frames of the multiple sensors includes: calculating the measurement indicators of Euclidean distance, intersection-over-union (IOU), speed distance and category distance between any two targets according to the target detection frames of the multiple sensors; setting weight coefficients of the measurement indicators, and weightedly summing each of the measurement indicators according to the weight coefficients to obtain a comprehensive distance; calculating the optimal matching result according to the comprehensive distance to obtain the optimal matching result; and setting a label for each target pair according to the optimal matching result.

[0070] In the specific implementation, the Euclidean distance is first calculated. The Euclidean distance is a common indicator for measuring the straight-line distance between two targets in space. For multi-target association, calculating the Euclidean distance between targets can preliminarily determine their proximity in space. Specifically, the calculation formula is The smaller the Euclidean distance, the closer the two targets are in space and the higher the possibility of association.

[0071] Then calculate IOU (Intersection over Union), which is mainly used to measure the degree of overlap between two regions. In multi-target association, the bounding box or region of the target can be regarded as two sets, and the ratio of their intersection to union is calculated. The calculation formula is Among them, S intersection is the intersection area of ​​the two target boxes, s union is the union area of ​​the two target boxes. The larger the IOR is, the higher the degree of overlap between the two targets and the greater the possibility of association.

[0072] Then calculate the speed distance, which can reflect the similarity of the target in motion state. By analyzing the historical data of the target trajectory, the speed vector of each target is calculated. For two targets, the difference d between their speed vectors is calculated. v =|V t -V s |, including the difference in speed and direction. The smaller the speed distance, the more similar the two targets are in motion, and they may be different observations of the same target.

[0073] Then calculate the category distance d type. The category distance is used to determine whether the category attributes of the targets are consistent. If the targets have different category labels, then the possibility of their association is usually low. For each target, determine the category it belongs to. If the categories of two targets are the same, the category distance is zero; if they are different, a certain distance value is assigned according to a pre-set rule to indicate their difference in category.

[0074] Finally, the Euclidean distance map is calculated. The original Euclidean distance d is converted to a value d between 0 and 1 0-1 , so that the change of Euclidean distance in different ranges can more intuitively reflect the relative closeness between targets.

[0075] The results of this mapping can be better combined with other distance indicators to improve the accuracy and reliability of multi-target association.

[0076] Furthermore, the weight coefficient of each evaluation index can be set. Different distance indicators have different importance in multi-target association. By setting the weight coefficient of each indicator, their contribution to the comprehensive distance can be adjusted. The setting of the weight coefficient can be based on experience, experimental results or prior knowledge. Optionally, in the embodiment of the present application, Euclidean distance and IOU are considered to be more important in the association, and larger weights W1=0.3 and W2=0.3 are assigned respectively; while for speed and category distance, which are relatively unreliable, smaller weights W3=0.1 and W4=0.1 are given.

[0077] Finally, the comprehensive distance is calculated. The comprehensive distance D is the result of weighted summation of each distance index according to the weight coefficient D = W1×d 0-1 +W2×IOU+W3×d v +W4×d type . By calculating the comprehensive distance, we can comprehensively consider the similarities of the targets in space, speed, category, etc., so as to more accurately judge whether they belong to different observations of the same target. For each target pair, their Euclidean distance, IOU, speed distance, category distance, etc. are multiplied by the corresponding weight coefficients, and then added to get the comprehensive distance. Smaller comprehensive distance values ​​indicate that the targets are more similar and more likely to belong to the same trajectory; larger comprehensive distance values ​​indicate that the targets are more different and less likely to belong to the same trajectory.

[0078] It can be understood that in addition to using the Hungarian algorithm to calculate the comprehensive distance, the correlation result can also be output, but the global optimality is not considered.

[0079] In one embodiment of the present application, the pre-trained target classification model includes: in the training phase of the model, calculating the value of the loss function according to the prediction results of the current model in each iteration; calculating the gradient of the loss function with respect to the model parameters through the back propagation algorithm; and updating the parameters of the model according to the gradient so that the value of the loss function is minimized.

[0080] The process of model training is to minimize the loss function by continuously adjusting the parameters of the model. XGBoost model training uses the gradient boosting algorithm to gradually improve the prediction results of the model by continuously adding new trees. In each iteration, calculating the loss and gradient back propagation are key steps. First, the value of the loss function is calculated based on the prediction results of the current model. Then, the gradient of the loss function to the model parameters is calculated through the back propagation algorithm. Finally, the parameters of the model are updated according to the gradient so that the value of the loss function gradually decreases.

[0081] In one embodiment of the present application, the input of a pre-trained target classification model in response to the metric of the target detection frame to obtain a target association result output by the pre-trained target classification model includes: loading the pre-trained target classification model; inputting a pre-trained target classification model in response to the metric of the target detection frame to predict and output a probability value of the result as a judgment on whether different target pairs belong to the same trajectory; inputting the probability value into the Hungarian algorithm to calculate the optimal match; after completing the calculation of the optimal match, obtaining the multi-sensor fusion trajectory target frame.

[0082] Enter the Hungarian algorithm to calculate the optimal match. The Hungarian algorithm is an efficient algorithm for solving the assignment problem. Enter the calculated comprehensive distance matrix into the Hungarian algorithm, which will find an optimal matching solution that minimizes the total matching distance.

[0083] According to the matching results of the Hungarian algorithm, a label is set for each target pair. If the two targets are matched successfully, that is, they belong to different observations of the same target, the label is set to 1; if they are not matched, the label is set to 0, indicating no match. The setting of labels provides a basis for subsequent manual screening and further analysis.

[0084] Preferably, the labeled samples are manually screened. The previous steps have obtained preliminary association results, but there may still be some errors or inaccurate matches. Therefore, manual screening is necessary. Manual screening can be performed by professionals to check the labeled samples one by one and adjust the labels according to the actual situation. For example, for some ambiguous situations, more contextual information, image features or other evidence can be combined to determine whether the target is truly matched. Manual screening can improve the accuracy and reliability of the association results and ensure that the final multi-target association results meet the needs of practical applications.

[0085] like Figure 3 The figure is a schematic diagram of the implementation principle of the target association method in the preferred embodiment of the present application.

[0086] Step 301, collecting multi-target association samples, step 302, training the XGBoost model, and step 303, deploying the XGBoost model. Through these steps, accurate association of multiple targets can be achieved, and the accuracy and efficiency of multi-target tracking can be improved.

[0087] The collection of multi-target associated samples in step 301 is further implemented as follows:

[0088] Step 3011, obtain the target detection tracking frame of multiple sensors. In the method used in the embodiment of the present application, the car is equipped with sensors such as on-board laser radar, camera, millimeter wave radar, and the parameters have been adjusted to a state where the surrounding environment perception information can be obtained in real time. Using the corresponding detection algorithm, the observation values ​​of the target frame can be obtained. These observation values ​​are in the form of 3D target frame corner coordinates (X s , Y s , Z s )={(x1, y1, z1), (x2, y2, z2),..., (x n ,y n , z n )}、Center point coordinates Speed ​​V s =(v x , v y , v z ), category type, and sensor attribute data structures.

[0089] Step 3012: Time alignment. In the process of multi-target tracking and association, time consistency is very important. The timestamp T of the target trajectory is t The timestamp T of the sensor s Alignment is performed to ensure that all objects are related in the same time coordinate system.

[0090] Step 3013: Use Kalman filtering to predict the target trajectory at the spatial position of the sensor. Kalman filtering can be used based on the historical state of the target and the current observation. and The future position of the target is predicted more accurately. By continuously updating the difference between the prediction and the observation, the target trajectory estimate is gradually corrected to make it closer to the actual situation, providing an accurate spatial basis for subsequent distance calculation and association.

[0091] Step 3014: Calculate the Euclidean distance. The Euclidean distance is a common indicator for measuring the linear distance between two targets in space. For multi-target association, calculating the Euclidean distance between targets can preliminarily determine their proximity in space. The calculation formula is: The smaller the Euclidean distance, the closer the two targets are in space and the higher the possibility of association.

[0092] Step 3014: Calculate IOU (intersection over union). The intersection over union is mainly used to measure the degree of overlap between two regions. In multi-target association, the bounding box or region of the target can be regarded as two sets, and the ratio of their intersection to union is calculated. The calculation formula is: Where S intersection is the intersection area of ​​the two target boxes, s union is the union area of ​​the two target boxes. The larger the IOR is, the higher the degree of overlap between the two targets and the greater the possibility of association.

[0093] Step 3015: Calculate the speed distance. The speed distance can reflect the similarity of the target in motion state. By analyzing the historical data of the target trajectory, the speed vector of each target is calculated. For two targets, the difference d between their speed vectors is calculated. v =|V t -V s |, including the difference in speed and direction. The smaller the speed distance, the more similar the two targets are in motion, and they may be different observations of the same target.

[0094] Step 3016: Calculate the category distance. Category distance d type Used to determine whether the category attributes of the targets are consistent. If the targets have different category labels, then the possibility of their association is usually low. For each target, determine the category it belongs to. If the categories of two targets are the same, the category distance is zero; if they are different, a certain distance value is assigned according to a pre-set rule to indicate their difference in category.

[0095] Step 3017: Euclidean distance mapping. The original Euclidean distance d is converted to a value d between 0 and 1 0-1Such a mapping enables the change of Euclidean distance in different ranges to more intuitively reflect the relative closeness between targets, which helps to better perform comprehensive calculations with other distance indicators and improve the accuracy and reliability of multi-target association.

[0096] Step 3018, set the weight coefficient of each evaluation index. Different distance indicators have different importance in multi-target association. By setting the weight coefficient of each indicator, their contribution to the comprehensive distance can be adjusted. This method considers that Euclidean distance and IOU are more important in association, and assigns larger weights W1=0.3 and W2=0.3 respectively; while speed and category distance are relatively unreliable, and are assigned smaller weights W3=0.1 and W4=0.1.

[0097] Step 3019: Calculate the comprehensive distance. The comprehensive distance D is the result of weighted summation of each distance index according to the weight coefficient D = W1×d 0-1 +W2×IOU+W3×d v +W4×d type By calculating the comprehensive distance, we can comprehensively consider the similarities of the targets in terms of space, speed, category, etc., so as to more accurately determine whether they belong to different observations of the same target. A smaller comprehensive distance value means that the targets are more similar and more likely to belong to the same trajectory; a larger comprehensive distance value means that the targets are more different and less likely to belong to the same trajectory.

[0098] Step 30100: Input the Hungarian algorithm to calculate the optimal match. The Hungarian algorithm is an efficient algorithm for solving the assignment problem. The calculated comprehensive distance matrix is ​​input into the algorithm, which will find an optimal matching solution to minimize the total matching distance.

[0099] Step 30111, set labels. According to the matching results of the Hungarian algorithm, set labels for each target pair. If the two targets are matched successfully, that is, they belong to different observations of the same target, the label is set to 1; if they are not matched, the label is set to 0, indicating no match. The setting of labels provides a basis for subsequent manual screening and further analysis.

[0100] Step 30112, manual screening. Although the previous steps have obtained preliminary association results, there may still be some errors or inaccurate matches. Therefore, manual screening is necessary. Professionals can check the labeled samples one by one and adjust the labels according to the actual situation. For example, for some ambiguous situations, more contextual information, image features or other evidence can be combined to determine whether the target is truly matched. Manual screening can improve the accuracy and reliability of the association results and ensure that the final multi-target association results meet the needs of actual applications.

[0101] The training XGBoost model of step 302 is further implemented as follows:

[0102] Step 3021, data preparation. Read the data set containing multi-target features and association results, and separate it into features and labels. The feature part covers various attributes of the target, such as Euclidean distance, intersection-over-union ratio, speed, category, etc. These features will be used as input to the model. The label is a label indicating whether the targets are associated, usually 0 (not associated) and 1 (associated).

[0103] Step 3022: Data segmentation. In order to evaluate the performance and generalization ability of the model, the data set is divided into a training set and a test set at a ratio of 0.8:0.2, where the number of samples in the training set is 34,570 and the number of samples in the test set is 6,914. This ensures that there is enough data for training and that there is a portion of independent data to verify the performance of the model on unseen data. The random segmentation method is used to ensure that the training set and the test set are representative.

[0104] Step 3023, data conversion. To improve the training efficiency and performance of the model, convert the data into XGBoost's DMatrix format. DMatrix is ​​a special data structure of XGBoost that can efficiently store and process large-scale data sets. When converting data, pass the features and target variables into the DMatrix constructor respectively so that XGBoost can correctly identify and process the data. In addition, DMatrix also supports parallel computing and memory optimization, which can greatly speed up the training of the model.

[0105] Step 3024, design the objective function. The objective function is the core of XGBoost model optimization. In the multi-objective association task, design an objective function suitable for the binary classification problem. The objective function usually consists of a loss function and a regularization term. The calculation formula is Where n is the number of samples, is the loss function, which is used to measure the difference between the model prediction results and the actual results, Ω(f k ) is a regularization term, which is used to control the complexity of the model and prevent the model from overfitting. The regularization term usually includes the number of leaf nodes and the sum of the squares of the leaf node weights: Where T is the number of leaf nodes, γ and λ are regularization parameters, and w j is the weight of the jth leaf node. For the binary classification problem, the binary cross entropy loss function is selected as the loss function, which can effectively measure the difference between the probability predicted by the model and the true label. At the same time, L1 and L2 regularization terms are added to control the complexity of the model and prevent overfitting.

[0106] Step 3025, design a loss function. In XGBoost, specify to use the binary cross entropy loss function. Binary cross entropy loss is a commonly used loss function, especially suitable for binary classification problems. Its calculation formula is: where y i is the true label, is the probability predicted by the model. The cross entropy loss function can measure the difference between the probability distribution predicted by the model and the true label. The closer the probability predicted by the model is to the true label, the smaller the cross entropy loss.

[0107] Step 3026, parameter setting. Setting appropriate parameters is crucial to the performance of the XGBoost model. In this step, the maximum depth of the tree is set to 4 to control the complexity of the model and prevent overfitting. The learning rate is set to 0.1, which determines the step size of the model to update the parameters in each iteration. A smaller learning rate can make the model more stable. Set the number of iterations to 200, which is the maximum number of times the model is trained. If the model does not improve significantly in 10 consecutive iterations, stop training early to save time and computing resources.

[0108] Step 3027, model training. The process of model training is to minimize the loss function by continuously adjusting the parameters of the model. In XGBoost, the gradient boosting algorithm is used for model training, and the prediction results of the model are gradually improved by continuously adding new trees. In each iteration, calculating the loss and gradient back propagation are key steps. First, the value of the loss function is calculated based on the prediction results of the current model. Then, the gradient of the loss function to the model parameters is calculated through the back propagation algorithm. Finally, the parameters of the model are updated according to the gradient to gradually reduce the value of the loss function.

[0109] The use of pre-labeling methods greatly improves the efficiency of labeling work. This method reduces the time and cost required for manual labeling through automation or semi-automation, while maintaining high standards of labeling quality. This not only speeds up the data processing process, but also provides richer and more accurate data support for subsequent model training and optimization.

[0110] Step 3028: Model prediction. Use the trained model to predict the test data. The prediction process is to input the features of the test data into the model to obtain the probability predicted by the model. Then, the probability is converted into a binary classification result. The threshold is usually set to 0.5. When the probability is greater than the threshold, the prediction result is 1 (correlation), otherwise it is 0 (no correlation).

[0111] Step 3029: Model evaluation. Model evaluation is used to measure the performance and generalization ability of the model. When calculating the evaluation index, the model prediction results are compared with the true labels, and the number of correctly predicted samples and the number of incorrectly predicted samples are counted.

[0112] Step 30210, save the best performing weight parameters. During the model training process, multiple different model parameter combinations may be obtained. In order to use the best performing model in subsequent applications, the best performing weight parameters need to be saved. By evaluating on the validation set, the best performing model parameter combination is selected and saved. Using the interface provided by XGBoost, the model parameters are saved as a file so that they can be loaded and used when needed, ensuring that the best performing model can be used in actual applications, improving the accuracy and efficiency of multi-target association.

[0113] The XGBoost model deployment in step 303 is further implemented as follows:

[0114] Step 3031, load the XGBoost model. In the deployment phase of multi-target association, you first need to load the trained XGBoost model. This step ensures that the model parameters obtained by previous training can be used for prediction and association calculations. When loading the model, you need to read the file storing the model parameters and restore these parameters to the model structure in memory.

[0115] Step 3032: Data preprocessing: Convert the original data into the DMatrix object required by XGBoost to ensure that the data can be correctly received by the model.

[0116] Step 3033: Output the model prediction results. After receiving the preprocessed input data, the model will make predictions and output the results. The prediction result is usually a probability value, indicating the possibility of association between targets. These probability values ​​reflect the model's judgment on whether different target pairs belong to the same trajectory.

[0117] Step 3034, set the threshold to 0.3. When the probability value is less than 0.3, the target pair is considered to be unrelated and will not proceed to the next step; when the probability value predicted by the model is greater than or equal to 0.3, the target pair is considered to be possibly related and is used as a distance measure to measure the degree of association between the two targets.

[0118] Step 3035: Input the Hungarian algorithm to calculate the optimal match. Input the predicted probability output by the model into the Hungarian algorithm to calculate the optimal match. The prediction result input to the Hungarian algorithm is usually a distance matrix or a cost matrix, in which each element represents the association cost between two targets. Convert the probability value predicted by the model into an association cost, using 1-probability value as the association cost. In this way, the higher the probability value, the lower the association cost, and the Hungarian algorithm is more likely to assign these target pairs together.

[0119] Step 3036: After the optimal match calculation is completed, the multi-sensor fusion trajectory target frame is obtained.

[0120] Experimental results:

[0121] The actual effect of the embodiment of the present application is illustrated by the test results on the self-designed data set. In the embodiment of the present application, the performance of the model is evaluated using evaluation indicators such as Accuracy, Precision, Recall, and F1 score. Accuracy refers to the proportion of the number of samples correctly predicted by the model to the total number of samples. It can measure the overall accuracy of the model. The calculation formula is: Precision, also known as the precision rate, refers to the proportion of true positive examples among all samples predicted as positive examples. The calculation formula is Recall refers to the ratio of the number of positive samples correctly predicted by the model to the actual number of positive samples. It can measure the model's ability to recognize positive samples. The calculation formula is: Among them, TP (True Positives) represents the number of samples that are actually positive and predicted as positive; TN (True Negatives) represents the number of samples that are actually negative and predicted as negative; FP (False Positives) represents the number of samples that are actually negative but predicted as positive; FN (False Negatives) represents the number of samples that are actually positive but predicted as negative. F1 score is the harmonic mean of precision and recall, which can comprehensively consider the accuracy and recall of the model. The calculation formula is: "↑" means the higher the better. As shown in Table 1 below, it lists the detailed comparison of the test results of the embodiment of the present application with the existing random forest method and MLP neural network method on the test set.

[0122] Table 1

[0123] Methods / Evaluation Indicators Accuracy↑ Precision↑ Recall↑ F1 score↑ Random Forest 0.976 0.910 0.845 0.878 MLP neural network method 0.978 0.930 0.840 0.885 In the present application embodiment 0.979 0.935 0.850 0.893

[0124] Accuracy comparison

[0125] In terms of accuracy, the accuracy of the MLP neural network method and the embodiment of the present application is relatively close and slightly higher than the random forest method. Among them, the accuracy in the embodiment of the present application is 0.979, which is slightly higher than 0.976 of the random forest method and 0.978 of the MLP neural network method. This shows that the overall correct classification ability of the sample in the embodiment of the present application is strong, and the category of the target can be judged more accurately.

[0126] Accuracy comparison

[0127] The precision reflects the proportion of true positive examples among the samples predicted as positive examples. The precision of the MLP neural network method is 0.930, which is further improved to 0.935 in the embodiment of the present application. The precision of the random forest method is relatively low, at 0.910. This shows that the embodiment of the present application is more accurate in determining positive samples, with fewer cases of misjudgment as positive examples, and can more reliably identify true positive examples.

[0128] Recall comparison

[0129] The recall rate measures the proportion of actual positive examples that are correctly predicted as positive examples. The recall rate in the embodiment of the present application is 0.850, which is higher than 0.840 of the MLP neural network method and 0.845 of the random forest method. This means that the embodiment of the present application can better discover actual positive samples and reduce missed detections, which has certain advantages for application scenarios that require a high recall rate.

[0130] F1 score comparison

[0131] F1 score is the harmonic mean of precision and recall, which takes into account the accuracy and comprehensiveness of the model. The F1 score in the embodiment of the present application is 0.893, which is higher than 0.885 of the MLP neural network method and 0.878 of the random forest method. This shows that the embodiment of the present application performs well in balancing precision and recall, and can maximize the coverage of positive samples while ensuring high accuracy.

[0132] In general, the embodiments of the present application have shown certain advantages in multiple indicators such as accuracy, precision, recall rate and F1 score. Compared with the random forest method and the MLP neural network method, the embodiments of the present application can classify more accurately, reduce misjudgments and missed detections, and have better performance and reliability. This is due to the innovations in algorithm design, feature selection and model optimization in the embodiments of the present application, which enable it to better adapt to specific problem scenarios and improve the accuracy and efficiency of multi-target association.

[0133] The embodiment of the present application also provides a target detection method, wherein the target association method is adopted, and the target association method includes:

[0134] Get target detection frames from multiple sensors;

[0135] Determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors;

[0136] In response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and a label of the target detection box, and the label indicates whether the targets are associated.

[0137] In some embodiments, the sensor includes at least one of the following: a laser radar, a camera, and a millimeter wave radar, and obtaining target detection frames of multiple sensors includes:

[0138] The corner point coordinates, center point coordinates, speed information, category information and sensor attribute information of the 3D target box of each target detection box in multiple sensors are obtained respectively.

[0139] In some embodiments, after acquiring the target detection frames of multiple sensors, the method further includes:

[0140] Generate target trajectory and target detection tracking frame according to the associated target detection frame;

[0141] Align the timestamp of the target trajectory with the timestamp of the sensor, perform target association in the same time coordinate system, and obtain the historical target detection and tracking frame;

[0142] The Kalman filter is used to predict the spatial position of the target trajectory in the sensor to obtain a predicted target detection tracking frame;

[0143] According to the historical target detection and tracking frame and the predicted target detection and tracking frame, a fused trajectory target prediction frame is output.

[0144] In some embodiments, determining the metric of the target detection frame according to the target detection frames of the multiple sensors includes:

[0145] Calculate the Euclidean distance, intersection over union (IOU), speed distance, and category distance metrics between any two targets based on the target detection frames of the multiple sensors;

[0146] Setting a weight coefficient of the measurement indicator, and performing weighted summation on each measurement indicator according to the weight coefficient to obtain a comprehensive distance;

[0147] Calculate the optimal matching result according to the comprehensive distance to obtain the optimal matching result;

[0148] According to the optimal matching result, a label is set for each target pair.

[0149] In some embodiments, the pre-trained target classification model includes:

[0150] In the training phase of the model, the value of the loss function is calculated based on the prediction results of the current model in each iteration;

[0151] Calculate the gradient of the loss function with respect to the model parameters through the back-propagation algorithm;

[0152] Update the model parameters according to the gradient to minimize the value of the loss function.

[0153] In some embodiments, the inputting of the metric indicator of the target detection box into a pre-trained target classification model to obtain the target association result output by the pre-trained target classification model includes:

[0154] Loading the pre-trained target classification model;

[0155] In response to the metric of the target detection box, a pre-trained target classification model is input to predict and output a probability value of the result as a judgment of whether different target pairs belong to the same trajectory;

[0156] Input the probability value into the Hungarian algorithm to calculate the optimal match;

[0157] After completing the calculation of the optimal match, obtain the multi-sensor fusion trajectory target frame.

[0158] The present application embodiment also provides a target association device 400, such as Figure 4 As shown, a schematic diagram of the structure of a target association device in an embodiment of the present application is provided, wherein the target association device 400 at least includes: an acquisition module 410, a determination module 420, and a response module 430, wherein:

[0159] In one embodiment of the present application, the acquisition module 410 is specifically used to: acquire target detection frames of multiple sensors.

[0160] Obtain the target detection and tracking results sensed by each of the multiple sensors on the vehicle to obtain the target detection frames of the multiple sensors. It should be noted that the sensors installed on the vehicle include but are not limited to laser radar, camera, millimeter wave radar and other sensors, and the number is not specifically limited. Each sensor has its own target perception result and tracking result.

[0161] In one embodiment of the present application, the determination module 420 is specifically used to determine a measurement index of the target detection frame according to the target detection frames of the multiple sensors.

[0162] For the target detection frames obtained from multiple sensors, we can further calculate metrics such as Euclidean distance, IOU (Intersection over Union), speed distance, and category distance. More metrics are designed for multi-target association. By introducing multi-dimensional metrics such as Euclidean distance, IOU (Intersection over Union), speed, and category, we not only enrich the perspective of evaluation, but also significantly improve the comprehensiveness and accuracy of the measurement results. Multi-dimensional considerations can more accurately capture and analyze the complex relationships between targets, thereby providing more reliable decision support in practical applications.

[0163] In one embodiment of the present application, the response module 430 is specifically used to: respond to the measurement index of the target detection box input into a pre-trained target classification model, and obtain the target association result output by the pre-trained target classification model, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and the label of the target detection box, and the label indicates whether the targets are associated.

[0164] In the training phase of the model, the pre-trained target classification model is obtained by machine learning training using multiple sets of sample data. For the training sample data, each set of data in the multiple sets of sample data includes: the metric of the target detection frame, i.e., each specific metric, and the label of the target detection frame, i.e., the label of whether the targets are associated.

[0165] After the metric index of the target detection frame is input into the pre-trained target classification model, the target association result output by the pre-trained target classification model is obtained. The multi-dimensional information fusion representation of the target is realized, the limitations of artificial rules are avoided, and complex nonlinear relationships can be processed, thereby exerting its unique advantages in target recognition and association analysis.

[0166] It can be understood that the above-mentioned target association device can implement each step of the target association method provided in the above-mentioned embodiment, and the relevant explanations about the target association method are applicable to the target association device, which will not be repeated here.

[0167] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0168] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0169] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0170] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a target associated device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0171] Get target detection frames from multiple sensors;

[0172] Determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors;

[0173] In response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and a label of the target detection box, and the label indicates whether the targets are associated.

[0174] The above application Figure 2The method performed by the target association device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0175] The electronic device may also perform Figure 2 The method executed by the target association device in Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0176] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 2 The method performed by the target association device in the illustrated embodiment is specifically used to perform:

[0177] Get target detection frames from multiple sensors;

[0178] Determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors;

[0179] In response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and a label of the target detection box, and the label indicates whether the targets are associated.

[0180] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0182] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0184] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0185] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0186] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0187] It should also be noted that the terms "include", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0188] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0189] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A target association method, wherein: The method comprises: Get target detection frames from multiple sensors; Determining a measurement index of the target detection frame according to the target detection frames of the multiple sensors; In response to the measurement index of the target detection box being input into a pre-trained target classification model, a target association result output by the pre-trained target classification model is obtained, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection box and a label of the target detection box, and the label indicates whether the targets are associated.

2. The method of claim 1, wherein: The sensor includes at least one of the following: a laser radar, a camera, and a millimeter wave radar, and the step of obtaining target detection frames of multiple sensors includes: The corner point coordinates, center point coordinates, speed information, category information and sensor attribute information of the 3D target box of each target detection box in multiple sensors are obtained respectively.

3. The method of claim 2, wherein: After acquiring the target detection frames of the multiple sensors, the method further includes: Generate target trajectory and target detection tracking frame according to the associated target detection frame; Align the timestamp of the target trajectory with the timestamp of the sensor, perform target association in the same time coordinate system, and obtain the historical target detection and tracking frame; The Kalman filter is used to predict the spatial position of the target trajectory in the sensor to obtain a predicted target detection tracking frame; According to the historical target detection and tracking frame and the predicted target detection and tracking frame, a fused trajectory target prediction frame is output.

4. The method of claim 2, wherein: The determining, according to the target detection frames of the multiple sensors, a metric of the target detection frame comprises: Calculate the Euclidean distance, intersection over union (IOU), speed distance, and category distance metrics between any two targets based on the target detection frames of the multiple sensors; Setting a weight coefficient of the measurement indicator, and performing weighted summation on each measurement indicator according to the weight coefficient to obtain a comprehensive distance; Calculate the optimal matching result according to the comprehensive distance to obtain the optimal matching result; According to the optimal matching result, a label is set for each target pair.

5. The method of claim 1, wherein: The pre-trained target classification model comprises: During the training phase of the model, the value of the loss function is calculated based on the prediction results of the current model in each iteration; Calculate the gradient of the loss function with respect to the model parameters through the back-propagation algorithm; Update the model parameters according to the gradient to minimize the value of the loss function.

6. The method of claim 1, wherein: The step of inputting a pre-trained target classification model in response to the metric indicator of the target detection frame to obtain a target association result output by the pre-trained target classification model comprises: Loading the pre-trained target classification model; In response to the metric of the target detection box, a pre-trained target classification model is input to predict and output a probability value of the result as a judgment of whether different target pairs belong to the same trajectory; Input the probability value into the Hungarian algorithm to calculate the optimal match; After completing the calculation of the optimal match, obtain the multi-sensor fusion trajectory target frame.

7. A target detection method, wherein: The target association method as described in any one of claims 1 to 6 is adopted.

8. A target association device, wherein: The device comprises: An acquisition module is used to obtain target detection frames of multiple sensors; A determination module, configured to determine a measurement index of the target detection frame according to the target detection frames of the multiple sensors; A response module is used to respond to the measurement index of the target detection frame input into a pre-trained target classification model to obtain a target association result output by the pre-trained target classification model, wherein the pre-trained target classification model is obtained by machine learning training using multiple groups of sample data, and each group of data in the multiple groups of sample data includes: the measurement index of the target detection frame and the label of the target detection frame, and the label indicates whether the targets are associated.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 6.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 6.

Images