Target matching method and device, equipment, readable storage medium and program product
By combining the weighted calculation of feature distance and covariance matrix distance in the autonomous driving system and the motion relationship covariance matrix, the problem of low target matching accuracy in multi-sensor fusion is solved, and the matching accuracy and robustness are improved.
Patent Information
- Application Number
- CN202411181784.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-03
AI Technical Summary
In autonomous driving systems, during the multi-sensor fusion process, the target matching accuracy is not high, especially in complex scenarios where targets are dense and dynamically changing, traditional motion feature matching methods are prone to lead to matching errors.
By determining the characteristic distance and covariance matrix distance between the observation target and the track target, and performing weighted calculations, combining the motion relationship covariance matrix to describe the dynamic changes and uncertainty of the target, more accurate target matching is achieved.
It improves the accuracy and robustness of target matching, enhances the accuracy of system's environment perception and target recognition, and reduces matching errors in complex scenarios.
Smart Images

Figure CN119068222A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a target matching method, device, equipment, readable storage medium and program product. Background Art
[0002] In the field of autonomous driving, multi-sensor fusion is an indispensable part. In order to achieve high-precision environmental perception and target recognition, autonomous driving systems usually use multiple types of sensors, such as lidar, radar, cameras, etc. These sensors provide different information in their respective advantage areas. By fusing this information, the system's ability to perceive the environment and the accuracy of target recognition can be improved.
[0003] In the multi-sensor fusion process, target matching is a key step. The main task of target matching is to associate the same target from different sensors so that it can correctly correspond to the same object when fusing data. However, in the related art, target matching is achieved based on the motion characteristics of the target, but there is a problem of low target matching accuracy. Summary of the invention
[0004] Based on this, it is necessary to provide a target matching method, device, vehicle equipment, computer-readable storage medium and computer program product that can improve the target matching accuracy in response to the above technical problems.
[0005] In a first aspect, the present application provides a target matching method, comprising:
[0006] Determine the observation target to be matched and multiple existing track targets;
[0007] Determining a first target motion characteristic of the observed target and a second target motion characteristic of each of the target tracks;
[0008] Determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features;
[0009] A weighted calculation is performed on the characteristic distance and the covariance matrix distance, and a track target matching the observed target is determined from the plurality of track targets based on the weighted calculation result.
[0010] In one embodiment, determining the first target motion feature of the observed target includes:
[0011] Receiving the perception results and perception time of the observation target by multiple sensors;
[0012] Timestamp alignment processing is performed on the plurality of perception results based on each of the perception times to obtain a first target motion feature of the observed target at a normalized timestamp.
[0013] In one embodiment, the second target motion feature includes a second target position of the track target, and determining the characteristic distance between the observation target and each of the track targets according to the first target motion feature and the second target motion feature includes:
[0014] According to the first target position in the first target motion feature and the second target position in the second target motion feature, respectively calculating and determining the Euclidean distance between the observed target and each of the track targets;
[0015] The Euclidean distance is determined as the characteristic distance.
[0016] In one embodiment, determining the covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features includes:
[0017] Determining a first covariance matrix of the observed target according to the first target motion feature;
[0018] Determine, according to each of the second target motion characteristics, a second covariance matrix of the track target corresponding to each of the second targets;
[0019] Determining a target characteristic difference between the observed target and each of the track targets;
[0020] For each of the track targets, the covariance matrix distance between the track target and the observation target is determined according to the second covariance matrix of each of the track targets, the target characteristic difference and the first covariance matrix.
[0021] In one embodiment, determining a first covariance matrix of the observed target according to the first target motion feature includes:
[0022] Determining the number of detections of the observed target;
[0023] When it is determined according to the number of detections that the observed target is detected for the first time, obtaining an initial covariance matrix;
[0024] According to the first target motion feature, updating the initial statistical mean corresponding to the target motion feature to obtain the target statistical mean;
[0025] The initial covariance matrix is updated according to the target statistical mean to determine a first covariance matrix of the observed target.
[0026] In one embodiment, updating the initial statistical mean corresponding to the target motion feature according to the first target motion feature to obtain the target statistical mean includes:
[0027] Obtaining an initial statistical mean corresponding to a target motion feature, and determining an observation dimension corresponding to the first target motion feature;
[0028] The initial statistical mean is updated according to the first target motion feature and the observation dimension to obtain a target statistical mean.
[0029] In one embodiment, determining a first covariance matrix of the observed target according to the first target motion feature includes:
[0030] Determining the number of detections of the observed target;
[0031] When it is determined according to the number of detections that the observed target is not detected for the first time, obtaining a third covariance matrix determined when the observed target was last detected;
[0032] A first covariance matrix of the observed target is determined according to the first target motion feature, the number of detections and the third covariance matrix.
[0033] In one embodiment, determining the first covariance matrix of the observed target according to the first target motion feature, the number of detections and the third covariance matrix includes:
[0034] Determine a first weight coefficient and a second weight coefficient respectively according to the number of detections;
[0035] updating a sample covariance matrix according to the first target motion feature to obtain a covariance matrix updated based on the first target motion feature;
[0036] A first covariance matrix of the observation target is determined according to the first weight coefficient, the second weight coefficient, the covariance matrix and the third covariance matrix.
[0037] In a second aspect, the present application also provides a target matching device, comprising:
[0038] The target determination module is used to determine the observation target to be matched and the existing multiple track targets:
[0039] A feature extraction module, used to determine a first target motion feature of the observed target and a second target motion feature of each of the track targets;
[0040] A data processing module, used for determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion characteristics and each of the second target motion characteristics;
[0041] A matching module is used to perform weighted calculation on the characteristic distance and the covariance matrix distance, and determine a track target that matches the observed target from the multiple track targets based on the weighted calculation result.
[0042] In a third aspect, the present application further provides a vehicle device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0043] Determine the observation target to be matched and multiple existing track targets;
[0044] Determining a first target motion characteristic of the observed target and a second target motion characteristic of each of the target tracks;
[0045] Determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features;
[0046] A weighted calculation is performed on the characteristic distance and the covariance matrix distance, and a track target matching the observed target is determined from the plurality of track targets based on the weighted calculation result.
[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0048] Determine the observation target to be matched and multiple existing track targets;
[0049] Determining a first target motion characteristic of the observed target and a second target motion characteristic of each of the target tracks;
[0050] Determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features;
[0051] A weighted calculation is performed on the characteristic distance and the covariance matrix distance, and a track target matching the observed target is determined from the plurality of track targets based on the weighted calculation result.
[0052] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0053] Determine the observation target to be matched and multiple existing track targets;
[0054] Determining a first target motion characteristic of the observed target and a second target motion characteristic of each of the target tracks;
[0055] Determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features;
[0056] A weighted calculation is performed on the characteristic distance and the covariance matrix distance, and a track target matching the observed target is determined from the plurality of track targets based on the weighted calculation result.
[0057] The above-mentioned target matching method, device, vehicle equipment, computer-readable storage medium and computer program product determine the observation target to be matched and the existing multiple track targets; determine the first target motion feature of the observation target, and the second target motion feature of each target track; determine the characteristic distance and covariance matrix distance between the observation target and each track target based on the first target motion feature and each second target motion feature. When performing target matching on the observation target to be matched and the existing multiple track targets, on the basis of traditional motion feature matching, the motion relationship covariance matrix of the target is added. The motion relationship covariance matrix is used to describe the dynamic changes and uncertainty of the target in different dimensions, and the correlation and image similarity between the observation target and the track target in the matching process are comprehensively considered, so as to achieve more accurate target matching and improve matching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 An application environment diagram of a target matching method in an embodiment;
[0060] Figure 2 is a schematic flow chart of a target matching method in one embodiment;
[0061] Figure 3 1 is a flow chart of a method for determining a covariance matrix distance in one embodiment;
[0062] Figure 4 A schematic diagram of a flow chart of a method for determining a covariance matrix in one embodiment;
[0063] Figure 5 A schematic diagram of a flow chart of a method for determining a covariance matrix in another embodiment;
[0064] Figure 6 is a flow chart of a target matching method in another embodiment;
[0065] Figure 7 is a structural block diagram of a target matching device in one embodiment;
[0066] Figure 8 FIG. 4 is a diagram showing the internal structure of a vehicle device in one embodiment. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.
[0069] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0071] It should be noted that target matching is a key step in the multi-sensor fusion process. The main task of target matching is to associate the same target from different sensors so that they can correctly correspond to the same object when fusing data. Traditional target matching methods are mainly based on the motion characteristics of the target, such as position, speed, and orientation. However, relying solely on these features for target matching may lead to inaccuracies due to sensor errors and environmental changes, especially in complex scenes with dense and dynamically changing targets. Traditional methods are prone to matching errors, and relying solely on motion features for matching is easily affected by the reduced sensor perception performance under environmental changes, and the system robustness is poor.
[0072] In order to further improve the accuracy of target matching, a target matching method is proposed. On the basis of traditional motion feature matching, the motion relationship covariance of the target is added, and the uncertainty of the target motion relationship is described by the covariance matrix, and the correlation and image similarity of the two targets in the matching process are comprehensively considered. Specifically, the motion relationship covariance of the target can reflect the dynamic changes and uncertainties of the target in terms of position, velocity, acceleration, etc. The final matching score obtained after weighted processing can achieve more accurate target matching.
[0073] The target matching method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the vehicle terminal 102 is deployed with multiple sensors 104, which are used to sense the vehicle driving environment, and through fusion technology, the vehicle can better understand the surrounding environment. Among them, sensing the vehicle driving environment can be sensing other vehicles, pedestrians, bicycles, traffic signals, road signs and lane markings, etc. The multiple sensors include cameras, laser radars, millimeter wave radars, etc., and the type and number of the multiple sensors are not limited here. Figure 1 The sensors shown in are for illustration only. The data storage system can store data that the vehicle terminal 102 needs to process. The data storage system can be integrated on the vehicle terminal 102, or placed on a cloud server or other network server. The vehicle terminal 102 determines the observation target to be matched based on the perception results of the sensor, and obtains multiple existing track targets from the vehicle terminal 102 system; determines the first target motion feature of the observation target, and the second target motion feature of each target track; determines the characteristic distance and covariance matrix distance between the observation target and each track target based on the first target motion feature and each second target motion feature; performs weighted calculation on the characteristic distance and covariance matrix distance, and determines the track target that matches the observation target from multiple track targets based on the weighted calculation result.
[0074] In an exemplary embodiment, Figure 2 As shown, a target matching method is provided, which is applied to Figure 1Taking the vehicle terminal in the example as an example, the method includes the following steps 202 to 208. Among them:
[0075] Step 202: Determine the observation target to be matched and multiple existing track targets.
[0076] It should be noted that after the vehicle receives the multi-sensor results, the multi-sensor fusion system on the vehicle side can be divided into two categories of targets. One is the observation target, which comes from sensor perception, and the other is the track target, which is the observation at a historical moment and is considered to be credible after passing the entire system link.
[0077] The number of observation targets to be matched may be single or multiple. In actual application scenarios, when a vehicle is driving, the number of observation targets sensed by the sensor is generally multiple. In this embodiment, a single observation target is used as an example for description.
[0078] Exemplarily, the multi-sensor fusion system receives the observation targets sensed by each sensor, and determines the observation target currently entering the multi-sensor fusion system and the system continuously maintains multiple track targets. It can be understood that one track target corresponds to one target track, and the target track includes multiple continuous frames, each of which includes the motion characteristics of the track target.
[0079] Step 204, determining the first target motion characteristics of the observed target and the second target motion characteristics of each track target.
[0080] It should be noted that if the track target is an observed target at a historical moment, the motion feature types included in the first target motion feature of the observed target and the second target motion feature of the track target may be the same, and the second target motion feature of each track target may be directly obtained from the system.
[0081] The first target motion feature and the second target motion feature are high-dimensional motion features, which can also be understood as a feature set that describes the target motion in multiple dimensions. These dimensions include but are not limited to at least one or more of position, speed, acceleration, direction, posture, etc. The position may include x, y, and z coordinates, and the speed may include horizontal and vertical speeds. The target perceived by multiple sensors includes at least one. In the case where multiple sensors perceive multiple observation targets, how to correctly associate the observation data measured by different sensors to the same observation target, its specific implementation method can be achieved through existing methods, which will not be repeated here. The first target motion feature is determined based on the perception results of the observation target by multiple sensors. Optionally, the perception results of the same observation target are processed to obtain the target motion feature under a unified timestamp. The first target motion feature can also be called observation data.
[0082] Exemplarily, by correctly associating observation data measured by different sensors to the same observation target, preprocessing the observation data associated with the same observation target to obtain the first target motion feature at a unified timestamp, and obtaining the second target motion feature of each track target from the vehicle. Processing the perception results of the same observation target to obtain the target motion feature at a unified timestamp can be preprocessing the perception results, Kalman filtering and timestamp alignment.
[0083] Preprocessing may include time synchronization, data cleaning, feature extraction, and data interpolation processing, and its specific implementation method can be achieved through existing methods and will not be elaborated here.
[0084] Step 206, determining the characteristic distance and covariance matrix distance between the observed target and each track target based on the first target motion feature and each second target motion feature.
[0085] The first target motion feature and the second target motion feature are both high-dimensional motion features, which may include but are not limited to at least one or more of position, speed, acceleration, direction, posture, etc. The feature distance may be determined by using Euclidean distance calculation for at least one motion feature of the same dimension in the first target motion feature and the second target motion feature. The first target motion feature includes each detected motion feature.
[0086] The covariance matrix distance can be determined based on the first covariance matrix of the observed target, the second covariance matrix of the track target, and the motion feature difference between the first target motion feature of the observed target and the second target motion feature of the track target. The first covariance matrix can be determined based on all motion features in the first target motion feature. The second covariance matrix of the track target can be obtained directly from the server, which can be on the vehicle side, a cloud server, or other network servers.
[0087] It should be noted that the covariance matrix can provide the ability to describe the instability of the target and quantify the uncertainty of the target's location, speed, acceleration and other multi-dimensional features, thereby providing more comprehensive information for target matching. In the process of target matching, the motion feature relationship and the motion relationship covariance can be combined to calculate the matching score through weighted processing to improve the accuracy and robustness of the matching.
[0088] Step 208 , performing weighted calculation on the characteristic distance and the covariance matrix distance, and determining a track target that matches the observed target from a plurality of track targets based on the weighted calculation result.
[0089] It should be noted that the weighted calculation of feature distance and covariance matrix distance can be understood as the weighted calculation of Euclidean distance and Mahalanobis distance. The weight here is an empirical value, which needs to be gradually adjusted later according to the operation of the entire system in the actual working environment. However, the weight is introduced here to adapt to the perception performance of different targets. For example, for smaller targets, the perception algorithm may not have good detection performance, and there is jitter, which leads to a larger Euclidean distance. However, the uncertainty between the motion relationships of these targets will not increase significantly. Here, the weight of the Mahalanobis distance can be increased, and the weight of the Euclidean distance can be reduced to obtain a better matching effect.
[0090] Exemplarily, a first matching value between the observed target and each track target is calculated based on the characteristic distance, and a second matching value between the observed target and each track target is calculated based on the covariance matrix distance. By weighted calculation of the first matching value and the second matching value, a final matching value between the observed target and each track target, i.e., a weighted calculation result, is obtained. A track target matching the observed target is determined from multiple track targets based on the weighted calculation result. The weighted calculation can be determined based on a preset weighted relationship.
[0091] It is understandable that if there are multiple matching track targets, the best track target can be further determined based on the characteristic distance or covariance matrix distance. For example, the track target corresponding to the smallest characteristic distance can be determined as the best track target. For another example, the track target corresponding to the smallest covariance matrix distance can be determined as the best track target. It can also be randomly selected. Here, there is no limitation on how to determine the best track target when there are multiple matching track targets.
[0092] In the above target matching method, the observation target to be matched and the existing multiple track targets are determined; the first target motion feature of the observation target and the second target motion feature of each target track are determined; according to the first target motion feature and each of the second target motion features, the characteristic distance and covariance matrix distance between the observation target and each of the track targets are determined. When the observation target to be matched and the existing multiple track targets are matched, the motion relationship covariance matrix of the target is added on the basis of traditional motion feature matching. The dynamic changes and uncertainty of the target in different dimensions are described by the motion relationship covariance matrix. The correlation and image similarity between the observation target and the track target in the matching process are comprehensively considered, so as to achieve more accurate target matching and improve matching accuracy.
[0093] Multiple sensors are deployed on the vehicle terminal. Since the acquisition frequency of each sensor may be different, in order for the multi-sensor system to more effectively integrate data from different sensors and provide more accurate and reliable target motion characteristics, it is necessary to unify the timestamps of the perception results of different sensors.
[0094] In an exemplary embodiment, determining a first target motion feature of an observed target includes:
[0095] Receive the perception results and perception times of the observed target from multiple sensors; perform timestamp alignment processing on the multiple perception results based on the perception times to obtain the first target motion feature of the observed target under the normalized timestamp.
[0096] Among them, different sensors have different acquisition frequencies, and the system receives the perception results of each sensor separately. It cannot guarantee that the trigger time perceived by each sensor is consistent, so a unified timestamp operation is required. The timestamp alignment process can be to interpolate the perception results between the perception time and the current time, and calculate the state information of the target at a specific time point, that is, the first target motion feature under the state under the normalized timestamp. The specific calculation method can be implemented by existing methods and will not be elaborated here. It should be noted that timestamp alignment processing of multiple perception results based on each perception time can provide more accurate and reliable target motion features. In this embodiment, the accuracy and robustness of target matching can be further improved.
[0097] In order to obtain the final matching score between the observed target and the track target, an accurate characteristic distance is required. Therefore, in an exemplary embodiment, the characteristic distance between the observed target and each track target is determined based on the first target motion feature and the second target motion feature, including: calculating and determining the Euclidean distance between the observed target and each track target based on the first target motion feature and the second target motion feature; and determining the Euclidean distance as the characteristic distance.
[0098] Among them, the feature difference in the same dimension is determined according to the first target motion feature and the second target motion feature, and each feature difference is summed and then squared, and the final Euclidean distance is determined as the feature distance. For example, according to the second target motion feature Y of the track target and the first target motion feature X of the observed target, the Euclidean distance D is calculated. E . The Euclidean distance calculation can be expressed as:
[0099]
[0100] Among them, m represents the number of feature dimensions.
[0101] It should be noted that for scenes with dense and dynamically changing targets, related technologies do not take into account the impact of matching errors caused by sensor instability when determining the Euclidean distance.
[0102] In the above embodiment, by using all motion features to calculate the Euclidean distance, a richer and more accurate target description can be provided, which helps to improve the accuracy of target recognition, tracking and prediction compared to using single-dimensional motion features.
[0103] Furthermore, when determining the characteristic distance, the Euclidean distance between the observed target and each track target can be calculated and determined according to the first target position in the first target motion feature and the second target position in the second target motion feature; and the Euclidean distance is determined as the characteristic distance. The specific implementation method can be achieved by the above method, which will not be described here. In this method, calculating the Euclidean distance according to the position is relative to calculating the Euclidean distance according to the speed, which can intuitively obtain the spatial distance between the targets and ensure the stability and reliability of the Euclidean distance.
[0104] In related technologies, when determining target matching, they all rely on the motion characteristics of the target. These characteristics may have errors and inconsistencies under the observation of different sensors, especially in complex scenes with dense targets and dynamic changes, which can easily lead to matching errors. In order to improve the matching accuracy on this basis, the motion relationship covariance is introduced for target matching. On this basis, in order to achieve matching, it is necessary to calculate the covariance matrix distance between the observed target and the track target.
[0105] Therefore, in an exemplary embodiment, if Figure 3 As shown, a method for determining covariance matrix distance is provided, comprising steps 302 to 308, wherein:
[0106] Step 302: Determine a first covariance matrix of the observed target according to the first target motion feature.
[0107] The first covariance matrix can be determined based on the first target motion feature, the attribute mean determined by the target track history frame, and the target observation number of the observed target. The calculation formula of the first covariance matrix can be as follows, where is the attribute mean calculated for the track history frame, X is the observed n-dimensional state, that is, the n-dimensional motion feature, and X is They are all matrices of shape (1, m), where n is the number of observations of the target.
[0108]
[0109] Step 304: Determine the second covariance matrix of the corresponding track target according to the motion characteristics of each second target.
[0110] The second covariance matrix of each track target is determined in the same manner as the first covariance matrix, which will not be described in detail here. In practical applications, the second covariance matrix of the track target can be directly obtained.
[0111] Step 306, determining the target characteristic difference between the observed target and each track target.
[0112] The first target motion feature of the observed target is subtracted from the second target motion feature of each track target to obtain the target feature difference between the observed target and each track target. For example, the first target motion feature X is subtracted from the second target motion feature Y to obtain XY, that is, the target feature difference.
[0113] Step 308, for each track target, determine the covariance matrix distance between the track target and the observed target according to the second covariance matrix, the target characteristic difference and the first covariance matrix of each track target.
[0114] Among them, according to the second covariance matrix, target characteristic difference and first covariance matrix of each track target, the covariance matrix distance between the track target and the observed target is determined, which can be determined by the value obtained by taking the square root of the product of the inverse of the sum of the second covariance matrix and the first covariance matrix, the target characteristic difference transposition and the target characteristic difference. The covariance matrix distance D M Specifically, it can be expressed as:
[0115]
[0116] Among them, ∑X represents the first covariance matrix, ∑ Y represents the second covariance matrix, XY represents the target feature difference, (XY) T Represents the transpose of the target feature difference.
[0117] Exemplarily, for each track target, the covariance matrix distance between each track target and the observed target is determined based on the second covariance matrix of each track target, the target characteristic difference between each track target and the observed target, and the first covariance matrix of the observed target.
[0118] In the above embodiment, the covariance matrix distance between the track target and the observed target is determined based on the second covariance matrix of the track target, the target characteristic difference between the track target and the observed target, and the first covariance matrix of the observed target, taking into account the dynamic characteristics and uncertainty of the target, thereby improving the accuracy and reliability of target tracking and state estimation.
[0119] It should be noted that in the matching process, for the generation of new targets, since the sensor still has stable tracking and the target state has not converged, the matching performance is low, resulting in poor matching robustness. Therefore, the initial covariance matrix is set based on prior knowledge.
[0120] In an exemplary embodiment, Figure 4 As shown, a method for determining a covariance matrix is provided, comprising steps 402 to 408. Wherein:
[0121] Step 402, determining the number of detections of the observed target.
[0122] The number of detections can also be understood as the number of observations or the tracking length.
[0123] Step 404, when it is determined that the observed target is detected for the first time according to the number of detections, an initial covariance matrix is obtained.
[0124] The initial covariance matrix can be obtained through statistics of the offline perception model, that is, for the first detection stage of the target, the initial covariance matrix is obtained in advance based on the performance statistics of the perception algorithm of each modality. In other words, a reasonable initial covariance matrix is set for each sensor to detect the target based on the characteristics and performance of each sensor perception algorithm.
[0125] Exemplarily, for the observed target in the first detection stage, the offline perception results of each modality perception algorithm are counted separately, and the target motion features in the overall offline results are used to calculate the initial covariance matrix of the modality.
[0126] Step 406: Update the initial statistical mean corresponding to the target motion feature according to the first target motion feature to obtain the target statistical mean.
[0127] It is understandable that the covariance matrix is calculated based on the statistical mean to obtain the uncertainty state of each attribute of the target. For example, the location, speed and other attributes of the target within 5m of a certain vehicle category are counted in advance in n samples to obtain the initial statistical mean, and the new statistical mean is calculated based on the initial statistical mean and the observation data obtained by the initial observation, that is, the target statistical mean. It should be noted that the statistical mean corresponding to the target motion feature is a vector.
[0128] Optionally, the target statistical mean can be achieved in the following manner: by obtaining an initial statistical mean corresponding to the target motion feature and determining an observation dimension corresponding to the first target motion feature; updating the initial statistical mean according to the first target motion feature and the observation dimension to obtain the target statistical mean.
[0129] The initial statistical mean can be determined by statistics based on the offline model. The target statistical mean μ′ can be specifically expressed as:
[0130]
[0131] Among them, X is the m-dimensional observation of the first frame, and μ is the initial statistical mean.
[0132] Step 408, updating the initial covariance matrix according to the target statistical mean to determine the first covariance matrix of the observed target.
[0133] Among them, the initial covariance matrix is updated according to the target statistical mean, and the first covariance matrix of the observed target can be expressed as:
[0134]
[0135] Among them, Σ′ is the initial covariance matrix obtained by the offline perception model statistics, X is the m-dimensional observation of the first frame, and n represents the feature dimension.
[0136] In the above embodiment, for the newly detected observation target, a relatively reliable covariance matrix of the target is obtained based on the target state calculation of the first frame, that is, the initial covariance matrix is set using prior knowledge, so that the newly generated target has higher accuracy in the initial matching stage, which overcomes the defect that the newly generated target lacks stable motion relationship covariance data. Compared with the related technology that all calculations are based on real-time motion state data and cannot represent the true state of the observation target, it can represent the true state of the observation target and improves the matching accuracy.
[0137] In the above embodiment, the method for determining the covariance matrix for the generation of new targets also exists in practical applications. In order to improve the matching accuracy and robustness, in an exemplary embodiment, Figure 5 As shown, a method for determining a covariance matrix is provided, including steps 502 to 506.
[0138] in:
[0139] Step 502, determining the number of detections of the observed target.
[0140] Step 504, when it is determined based on the number of detections that the observed target is not detected for the first time, obtain a third covariance matrix determined when the observed target was last detected.
[0141] If the number of detections is 2, the third covariance matrix determined when the observed target was detected last time is the covariance matrix determined in the initial detection. Starting from the second detection, the covariance matrix is determined in the same way for each subsequent detection.
[0142] Step 506, determining a first covariance matrix of the observed target according to the first target motion feature, the number of detections and the third covariance matrix.
[0143] Specifically, the offline perception results of each modal perception algorithm are counted separately, and the target motion features in the overall offline results are used to calculate the initial covariance matrix of the modality. The dynamic weight formula is used together with the observed first target motion features and the number of detections to calculate the motion feature covariance matrix to obtain the final covariance matrix, that is, the first covariance matrix of the observed target.
[0144] It should be noted that as the target is subsequently detected, the influence of the initial matrix gradually decreases to ensure the stability of the target matching algorithm. In the subsequent detection of the target, in order to obtain a more realistic target motion, in an exemplary embodiment, a method for determining the first covariance matrix of the observed target is provided, specifically including:
[0145] The first weight coefficient and the second weight coefficient are determined according to the number of detections; the sample covariance matrix is updated according to the first target motion feature to obtain the updated covariance matrix based on the first target motion feature; the first covariance matrix of the observed target is determined according to the first weight coefficient, the second weight coefficient, the covariance matrix and the third covariance matrix.
[0146] Among them, in the subsequent detection of the target, in order to obtain a more realistic target motion situation, the following weight calculation formula is used to calculate the stable covariance matrix ∑ n , that is, the first covariance matrix, which can be specifically expressed as:
[0147] ∑ n =t (1-n) Cov(∑′,X)+(1-t (1-n) )∑ n-1
[0148] Where t is the calibration value, which is adjusted according to the actual state. n is the number of times the observed target is detected, n is greater than or equal to 2, ∑′ is the initial covariance matrix, X is the first target motion feature, ∑ n-1 The last determined covariance matrix. (1-n) is the first weight coefficient, (1-t (1-n) ) is the second weight coefficient.
[0149] It should be noted that the first covariance matrix ∑ nOne part is obtained through pre-statistics, and the other part is calculated through perception during system operation. However, when the target is observed for a short time, the sample size is small, and the calculated covariance matrix is unstable and cannot accurately reflect the actual relationship. For this reason, the pre-statistical covariance matrix is added, but at the same time, in order to reduce the impact of pre-statistics in the case of long-term observation, its weight is reduced to obtain authenticity.
[0150] In the above embodiment, for the first detection stage of the target, the performance statistics of the perception algorithm for each modality are used to obtain the initial covariance matrix, and a relatively reliable covariance matrix of the observed target is calculated in the first frame target state. As the detection of the target stabilizes, the initial matrix weight is gradually reduced to obtain a stable covariance matrix that finally participates in the matching algorithm. In other words, the initial covariance weight is dynamically adjusted. As the target detection gradually stabilizes, the real-time detection data provides more accurate target motion state information than the initial covariance matrix, thereby ensuring the robustness and accuracy of the matching algorithm.
[0151] In another exemplary embodiment, Figure 6 As shown, a target matching method is provided, which is applied to Figure 1 Taking the vehicle terminal in the example as an example, the method includes the following steps 202 to 208. Among them:
[0152] Step 602, receiving the observed target output by the multiple sensors, as well as the perception result and perception time of the observed target.
[0153] Step 604: perform timestamp alignment processing on multiple perception results based on each perception time to obtain a first target motion feature of the observed target at a normalized timestamp.
[0154] Step 606, determining the first target motion characteristic of the observed target and the second target motion characteristic of each track target in the existing multiple track targets.
[0155] Step 608, determining the characteristic distance between the observed target and each track target based on the first target motion characteristic and each second target motion characteristic.
[0156] Step 610, determine whether the observed target is detected for the first time, if it is detected for the first time, execute step 612, otherwise, execute step 614.
[0157] Step 612, obtaining an initial covariance matrix, and determining a first covariance matrix of the observed target according to the first target motion feature and the initial covariance matrix.
[0158] Among them, when the observed target is detected for the first time, the first covariance matrix can be determined by: obtaining the initial covariance matrix; updating the initial statistical mean corresponding to the target motion feature according to the first target motion feature to obtain the target statistical mean; updating the initial covariance matrix according to the target statistical mean to determine the first covariance matrix of the observed target.
[0159] Step 614, determining a first covariance matrix of the observed target based on the first target motion feature, the number of detections, and the third covariance matrix determined at the last detection.
[0160] Among them, when the observed target is not detected for the first time, the first covariance matrix can be determined by: obtaining the third covariance matrix determined when the observed target was detected for the last time, determining the first weight coefficient and the second weight coefficient according to the number of detections, respectively, updating the sample covariance matrix according to the first target motion feature, obtaining the covariance matrix updated based on the first target motion feature, and determining the first covariance matrix of the observed target according to the first weight coefficient, the second weight coefficient, the covariance matrix and the third covariance matrix.
[0161] Step 616, determining the target characteristic difference between the observed target and each track target.
[0162] Step 618, for each track target, determine the covariance matrix distance between the track target and the observed target according to the second covariance matrix, the target characteristic difference and the first covariance matrix of each track target.
[0163] The method for determining the covariance matrix distance can be implemented in the above manner, which will not be described in detail here.
[0164] Step 620, performing weighted calculation on the characteristic distance and the covariance matrix distance, and determining a track target that matches the observed target from a plurality of track targets based on the weighted calculation result.
[0165] It should be noted that the specific implementation methods in this embodiment can all be implemented through the above methods, which will not be described in detail here.
[0166] In the above embodiment, multi-sensor fusion targets are combined with multi-dimensional motion relationship matching, and the covariance between target motion features is combined on the basis of traditional motion relationship matching, and weighted operations are performed to obtain a more stable and accurate target feature relationship for matching between targets. At the same time, for the first detection stage of the target, the performance statistics of the perception algorithm for each modality are used to obtain the initial covariance matrix, and the target state is calculated in the first frame to obtain a relatively reliable covariance matrix of the target. As the detection of the target stabilizes, the initial matrix weight is gradually reduced to make the system adapt to changes in the target state. The accuracy, stability and adaptability to complex environments of multi-sensor fusion target matching can be improved, thereby improving the accuracy and robustness of target matching.
[0167] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0168] Based on the same inventive concept, the embodiment of the present application also provides a target matching device for implementing the target matching method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target matching device embodiments provided below can refer to the limitations on the target matching method above, and will not be repeated here.
[0169] In an exemplary embodiment, Figure 7 As shown, a target matching device is provided, including: a target determination module 702, a feature extraction module 704, a data processing module 706 and a matching module 708, wherein:
[0170] The target determination module 702 is used to determine the observation target to be matched and the existing multiple track targets.
[0171] The feature extraction module 704 is used to determine the first target motion feature of the observed target and the second target motion feature of each track target.
[0172] The data processing module 706 is used to determine the characteristic distance and covariance matrix distance between the observed target and each track target according to the first target motion feature and each second target motion feature.
[0173] The matching module 708 is used to perform weighted calculation on the feature distance and the covariance matrix distance, and determine a track target that matches the observed target from multiple track targets based on the weighted calculation result.
[0174] The target matching device determines the observation target to be matched and multiple existing track targets; determines the first target motion feature of the observation target and the second target motion feature of each target track; determines the characteristic distance and covariance matrix distance between the observation target and each track target based on the first target motion feature and each second target motion feature. When performing target matching on the observation target to be matched and the multiple existing track targets, the motion relationship covariance matrix of the target is added on the basis of traditional motion feature matching. The motion relationship covariance matrix is used to describe the dynamic changes and uncertainty of the target in different dimensions, and the correlation and image similarity between the observation target and the track target in the matching process are comprehensively considered, thereby achieving more accurate target matching and improving matching accuracy.
[0175] Optionally, in an exemplary embodiment, the feature extraction module 704 is further used to receive the perception results and perception time of the multi-sensor on the observed target;
[0176] Timestamp alignment is performed on multiple perception results based on each perception time to obtain the first target motion feature of the observed target at a normalized timestamp.
[0177] Optionally, in an exemplary embodiment, the data processing module 706 is also used to calculate and determine the Euclidean distance between the observed target and each track target based on the first target position in the first target motion feature and the second target position in the second target motion feature; and determine the Euclidean distance as the characteristic distance.
[0178] Optionally, in an exemplary embodiment, the data processing module 706 is further configured to determine a first covariance matrix of the observed target according to the first target motion feature;
[0179] Determine the second covariance matrix of the corresponding track target according to the motion characteristics of each second target;
[0180] Determine the target characteristic difference between the observed target and each track target;
[0181] For each track target, the covariance matrix distance between the track target and the observation target is determined according to the second covariance matrix, the target characteristic difference and the first covariance matrix of each track target.
[0182] Optionally, in an exemplary embodiment, the target matching device includes a covariance matrix determination module, and the covariance matrix determination module is used to determine the number of detections of the observed target;
[0183] When the observed target is determined to be detected for the first time according to the number of detections, an initial covariance matrix is obtained;
[0184] According to the first target motion feature, an initial statistical mean corresponding to the target motion feature is updated to obtain a target statistical mean;
[0185] The initial covariance matrix is updated according to the target statistical mean to determine the first covariance matrix of the observed target.
[0186] Optionally, in an exemplary embodiment, the target matching device includes an updating module, the updating module is used to obtain an initial statistical mean corresponding to the target motion feature, and determine an observation dimension corresponding to the first target motion feature;
[0187] The initial statistical mean is updated according to the first target motion feature and the observation dimension to obtain the target statistical mean.
[0188] Optionally, in an exemplary embodiment, the covariance matrix determination module is used to determine the number of detections of the observed target;
[0189] When it is determined according to the number of detections that the observed target is not detected for the first time, obtaining a third covariance matrix determined when the observed target was last detected;
[0190] A first covariance matrix of the observed target is determined according to the first target motion feature, the number of detections and the third covariance matrix.
[0191] Optionally, in an exemplary embodiment, the covariance matrix determination module is further used to determine the first weight coefficient and the second weight coefficient according to the number of detections;
[0192] Updating the sample covariance matrix according to the first target motion feature to obtain a covariance matrix updated based on the first target motion feature;
[0193] A first covariance matrix of the observed target is determined according to the first weight coefficient, the second weight coefficient, the covariance matrix and the third covariance matrix.
[0194] Each module in the above target matching device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a vehicle device in the form of hardware, or can be stored in a memory in the vehicle device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0195] In an exemplary embodiment, a vehicle device is provided. The vehicle device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The vehicle equipment includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the vehicle equipment is used to provide computing and control capabilities. The memory of the vehicle equipment includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the vehicle equipment is used to exchange information between the processor and an external device. The communication interface of the vehicle equipment is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a target matching method is implemented. The display unit of the vehicle equipment is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the vehicle equipment can be a touch layer covering the display screen, or a button, trackball or touchpad set on the vehicle equipment shell, or an external keyboard, touchpad or mouse.
[0196] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the vehicle equipment to which the scheme of the present application is applied. The specific vehicle equipment may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0197] In an exemplary embodiment, a vehicle device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0198] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0199] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0200] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0201] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0202] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0203] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A target matching method, characterized in that: The method comprises: Determine the observation target to be matched and multiple existing track targets; Determining a first target motion characteristic of the observed target and a second target motion characteristic of each of the track targets; Determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features; A weighted calculation is performed on the characteristic distance and the covariance matrix distance, and a track target matching the observed target is determined from the plurality of track targets based on the weighted calculation result.
2. The method according to claim 1, characterized in that The determining of the first target motion feature of the observed target includes: Receiving the perception results and perception time of the observation target by multiple sensors; Timestamp alignment processing is performed on the plurality of perception results based on each of the perception times to obtain a first target motion feature of the observed target at a normalized timestamp.
3. The method according to claim 1, characterized in that The second target motion feature includes a second target position of the track target, and determining the characteristic distance between the observation target and each of the track targets according to the first target motion feature and the second target motion feature includes: According to the first target position in the first target motion feature and the second target position in the second target motion feature, respectively calculating and determining the Euclidean distance between the observed target and each of the track targets; The Euclidean distance is determined as the characteristic distance.
4. The method according to claim 1, characterized in that: Determining the covariance matrix distance between the observed target and each of the track targets according to the first target motion feature and each of the second target motion features includes: Determining a first covariance matrix of the observed target according to the first target motion feature; Determine, according to each of the second target motion characteristics, a second covariance matrix of the track target corresponding to each of the second targets; Determining a target characteristic difference between the observed target and each of the track targets; For each of the track targets, the covariance matrix distance between the track target and the observation target is determined according to the second covariance matrix of each of the track targets, the target characteristic difference and the first covariance matrix.
5. The method according to claim 4, characterized in that The step of determining a first covariance matrix of the observed target according to the first target motion feature includes: Determining the number of detections of the observed target; When it is determined according to the number of detections that the observed target is detected for the first time, obtaining an initial covariance matrix; According to the first target motion feature, updating the initial statistical mean corresponding to the target motion feature to obtain the target statistical mean; The initial covariance matrix is updated according to the target statistical mean to determine a first covariance matrix of the observed target.
6. The method according to claim 5, characterized in that The updating, according to the first target motion feature, of the initial statistical mean corresponding to the target motion feature to obtain the target statistical mean includes: Obtaining an initial statistical mean corresponding to a target motion feature, and determining an observation dimension corresponding to the first target motion feature; The initial statistical mean is updated according to the first target motion feature and the observation dimension to obtain a target statistical mean.
7. The method according to claim 4, characterized in that The step of determining a first covariance matrix of the observed target according to the first target motion feature includes: Determining the number of detections of the observed target; When it is determined according to the number of detections that the observed target is not detected for the first time, obtaining a third covariance matrix determined when the observed target was last detected; A first covariance matrix of the observed target is determined according to the first target motion feature, the number of detections and the third covariance matrix.
8. The method according to claim 7, characterized in that Determining a first covariance matrix of the observed target according to the first target motion feature, the number of detections and the third covariance matrix includes: Determine a first weight coefficient and a second weight coefficient respectively according to the number of detections; updating a sample covariance matrix according to the first target motion feature to obtain a covariance matrix updated based on the first target motion feature; A first covariance matrix of the observation target is determined according to the first weight coefficient, the second weight coefficient, the covariance matrix and the third covariance matrix.
9. A target matching device, characterized in that: The device comprises: A target determination module is used to determine the observation target to be matched and multiple existing track targets; A feature extraction module, used to determine a first target motion feature of the observed target and a second target motion feature of each of the track targets; A data processing module, used for determining a characteristic distance and a covariance matrix distance between the observed target and each of the track targets according to the first target motion characteristics and each of the second target motion characteristics; A matching module is used to perform weighted calculation on the characteristic distance and the covariance matrix distance, and determine a track target that matches the observed target from the multiple track targets based on the weighted calculation result.
10. A vehicle device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
