Object association model training method, object association method and device

Abstract

The invention provides a training method of an object association model, and an object association method and device, and the method comprises the steps: obtaining a first state information sample of a first object sample collected by a vehicle at a T moment, and obtaining a second state information sample of at least one second object sample collected by the vehicle at a preset moment before the T moment; obtaining a target association matrix according to the first state information sample, the second state information sample and an object association model; the target incidence matrix is used for reflecting the similarity between the first state information sample and the second state information sample; obtaining an object association result sample according to the target association matrix, and training an object association model based on the object association result sample to obtain a trained object association model; the object association result sample comprises a first object sample and a target second object sample with the maximum similarity with the first object sample. The obtained object association model is high in accuracy and low in model training cost.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a training method for an object association model, an object association method and a device. Background Art

[0002] With the rapid development of autonomous driving perception technology, multi-sensor fusion perception has gradually become the main perception technology for vehicles with high precision and high performance. Raw perception data can be obtained through sensors, and then target tracking can be performed based on the raw perception data.

[0003] In related technologies, the target state can be predicted through a constant turning rate and speed model, and the target state prediction result and the current detection result can be associated and matched based on a metric function of multi-dimensional feature fusion. The multi-target association problem is converted into a weighted bipartite graph minimum full matching problem based on the Hungarian matching algorithm, and the association relationship between targets is represented by a two-dimensional cost matrix.

[0004] However, the method based on the correlation technology requires the construction of multiple metric functions and metric function weights, and the association process and association algorithm are complicated, resulting in low processing efficiency, and the parameters of different sensors need to be adjusted, which increases the development cost. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention provide a method for training an object association model, an object association method and a device that overcome the above problems or at least partially solve the above problems.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, an embodiment of the present application discloses a method for training an object association model, comprising: obtaining a first state information sample of a first object sample collected by a vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before the time T; the second object sample and the second state information sample correspond one to one; according to the first state information sample, the second state information sample and the object association model, a target association matrix is ​​obtained; the target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample; according to the target association matrix, an object association result sample is obtained, and the object association model is trained based on the object association result sample to obtain a trained object association model; the object association result sample includes the first object sample, and a target second object sample with the greatest similarity to the first object sample.

[0008] In a second aspect, an embodiment of the present application discloses an object association method, comprising: obtaining first state information of a first object collected by a vehicle at a current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment; the second object and the second state information correspond one-to-one; inputting the first state information sample and the second state information sample into a trained object association model to obtain a target second object associated with the first object; wherein the trained object association model is obtained by the object association model training method described in the first aspect.

[0009] In a third aspect, an embodiment of the present application discloses a training device for an object association model, comprising: a first acquisition module, used to acquire a first state information sample of a first object sample collected by a vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before the time T; the second object sample and the second state information sample correspond one to one; a second acquisition module, used to obtain a target association matrix based on the first state information sample, the second state information sample and the object association model; the target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample; a third acquisition module, used to obtain an object association result sample according to the target association matrix, and train the object association model based on the object association result sample to obtain a trained object association model; the object association result sample includes the first object sample, and a target second object sample with the greatest similarity to the first object sample.

[0010] In a fourth aspect, an embodiment of the present application discloses an object association device, comprising: a fourth acquisition module, used to acquire first state information of a first object collected by a vehicle at a current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment; the second object and the second state information correspond one to one; a fifth acquisition module, used to input the first state information sample and the second state information sample into a trained object association model to obtain a target second object associated with the first object; wherein the trained object association model is obtained by the object association model training method described in the first aspect.

[0011] In a fifth aspect, an embodiment of the present application discloses an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0012] In a sixth aspect, an embodiment of the present application discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0013] In a seventh aspect, an embodiment of the present application discloses a vehicle, comprising a training device for an object association model as described in the third aspect, an object association device as described in the fourth aspect, or an electronic device as described in the fifth aspect.

[0014] In this embodiment, a first state information sample of a first object sample collected by the vehicle at time T and a second state information sample of at least one second object sample collected by the vehicle at a historical moment before time T are obtained; a target association matrix is ​​obtained according to the first state information sample, the second state information sample and the object association model; an object association result sample is obtained according to the target association matrix sample, and the object association model is trained using the object association result sample to obtain a trained object association model; the object association result sample includes the first object sample and the target second object sample associated with the first object sample. The method of this embodiment can improve the prediction performance of the object association model by associating and fusing object samples at multiple moments. In addition, this embodiment does not need to train the object association model separately for sensors with different layouts using the state information collected by them, but can associate and fuse the state information of sensors with different installation layouts under the same object association model. Furthermore, this embodiment reduces the operations of logic setting, threshold setting, adjustment and screening of different sensors by manual methods, and can associate the tracking targets of all sensors through a shared object association model, thereby reducing development costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of the steps of an object association model training method provided by an embodiment of the present invention;

[0016] Figure 2 It is a structural schematic diagram of a target association matrix acquisition module in an object association model provided by an embodiment of the present invention;

[0017] Figure 3 is a flowchart of another object association model training method provided by an embodiment of the present invention;

[0018] Figure 4 is a flow chart of a method for obtaining a target association pair provided by an embodiment of the present invention;

[0019] Figure 5 is a flowchart of a method for associating objects provided by an embodiment of the present invention;

[0020] Figure 6 is a flowchart of the steps of another object association model training method provided by an embodiment of the present invention;

[0021] Figure 7 is a flowchart of another object association model training method provided by an embodiment of the present invention;

[0022] Figure 8 is a block diagram of an object association model training device provided by an embodiment of the present invention;

[0023] Fig. 9 is a block diagram of an object association device provided by an embodiment of the present invention;

[0024] Fig.10 is a block diagram of an electronic device provided in an embodiment of the present application;

[0025] Fig.11 This is a block diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0027] refer to Figure 1 , which shows the object association model training method provided by the embodiment of the present application, the method includes:

[0028] Step 101 , obtaining a first state information sample of a first object sample collected by a vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before time T.

[0029] The second object samples and the second state information samples correspond one to one.

[0030] For example, the first object sample and the second object sample may be objects in front of the vehicle or in the surrounding environment collected by the vehicle sensor. For example, the first object sample and the second object sample may be vehicles, pedestrians, bicycles, or other objects collected by the vehicle sensor. The first object may include at least one object in front of the vehicle collected by the vehicle sensor at time T, and the second object may include at least one object in front of the vehicle collected by the vehicle sensor at a preset time before time T.

[0031] The type of state information included in the first state information sample may be the same as the type of state information included in the second state information sample. For example, the first state information sample and the second state information sample may both include target position, target speed, target acceleration, target frame size, heading angle, target category, sensor category of the sensor used to collect state information, etc. The target frame size is the size of the target frame (e.g., a rectangular frame) used to represent the object.

[0032] There may be many sensors for collecting the first state information sample and the second state information sample, for example, at least one of the following may be included: laser radar, camera, millimeter wave radar, etc. In practical applications, sensors are the main tools for autonomous vehicles to perceive the real environment, and different types of sensors have different performances in state information collection. For example, visual perception sensors include cameras, etc. Visual sensors can obtain rich semantic information, as well as information such as the color, shape, and texture of the target object. However, the camera cannot read the target depth position information, so it is not suitable for estimating the 3D position of the target object. Laser radar has the characteristics of high resolution and high precision, and can accurately reflect the 3D position information of the target object, but it has the disadvantages of low anti-interference ability and high cost. Millimeter wave radar is less affected by weather interference, and has strong detection and tracking capabilities for targets, but its detection distance is relatively short.

[0033] For example, the first state information sample s can be expressed as Among them, n s is the number of sensor output targets, R represents the feature, and 11 represents the type of state information included in the first state information sample. For example, the type of the first state information may include at least one of the following: target location Target speed Target acceleration Target box size (l s ,w s ), heading angle θ s , target category c s , sensor categories s .in, and are the positions of the X-axis and Y-axis of the vehicle coordinate system, and are the velocity components of the X-axis and Y-axis of the vehicle coordinate system, and are the acceleration classifications of the X-axis and Y-axis of the vehicle coordinate system, respectively. s and w s are the length and width of the target box respectively.

[0034] The second state information sample f iIt can be expressed as Among them, n f Output the number of targets for target association, i=1, 2, ..., t; 11 represents the type of state information included in the second state information sample, and R represents the feature. For example, the second state information may include at least one of the following: object position Target speed Target acceleration Target box size (l f ,w f ), heading angle θ f , target category c f , sensor categories f .in, and are the positions of the X-axis and Y-axis of the vehicle coordinate system, and are the velocity components of the X-axis and Y-axis of the vehicle coordinate system, and are the acceleration classifications of the X-axis and Y-axis of the vehicle coordinate system, respectively. f and w f are the length and width of the target box respectively.

[0035] For example, the state information collected by multiple sensors at a preset time can be obtained, and the same type of state information collected by different sensors is weighted and fused to obtain multi-sensor fusion target data, and the multi-sensor fusion target data is determined as the second state information sample in this step. For example, the sensor may include a visual sensor, a millimeter wave radar sensor, or a laser radar. If the three sensors all collect the location information of the target object, the location information of the three sensors is weighted and fused to obtain the object position in the second state information.

[0036] Step 102: Obtain a target association matrix according to the first state information sample, the second state information sample and the object association model.

[0037] The target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample.

[0038] For example, a first state feature of a first state information sample and a second state feature of a second state information sample are obtained, and a target association matrix is ​​obtained according to the first state feature, the second state feature, and a matrix multiplication operation mechanism in an object association model.

[0039] Furthermore, the object association model includes Figure 2 The target association matrix calculation module shown in FIG. 1 obtains the first state feature of the first state information sample and uses it as Figure 2The sensor target characteristic shown in FIG. 1 is obtained by obtaining the second state characteristic of the second state information sample and using it as Figure 2 The multi-sensor fusion target features shown in the figure. The first state feature is processed by dimension increase to obtain the first dimension increase feature, the second state feature is processed by dimension increase to obtain the second dimension increase feature, the first dimension increase feature and the second dimension increase feature are processed by matrix multiplication to obtain the association weight matrix, and the association weight matrix is ​​processed by dimension reduction to obtain the target association matrix.

[0040] Step 103: Obtain object association result samples according to the target association matrix, and train an object association model based on the object association result samples to obtain a trained object association model.

[0041] The object association result samples include a first object sample and a target second object sample having the greatest similarity to the first object sample.

[0042] The target association matrix includes a plurality of matrix elements, each of which represents the similarity between the first state information sample and the second state information sample. According to the target association matrix, a target second state information sample with the greatest similarity to the first state information sample is obtained from the second state information sample, and a second object sample corresponding to the target second state information sample is determined as the target second object sample with the greatest similarity to the first object sample.

[0043] In the related art, object association can be performed by association algorithms based on intersection over union (IoU) matching and association algorithms based on nearest neighbor algorithms. Specifically, the method based on IoU matching includes: calculating the IoU of the detection frame at the previous moment and all the detection frames at the current moment, and determining that it is associated when the IoU is greater than the threshold. The nearest neighbor algorithm includes: associating targets at different moments based on the distance metric between targets (Euclidean distance, Mahalanobis distance, etc.). However, these algorithms have the problem of low accuracy of association results.

[0044] In related technologies, the multi-target association problem can also be transformed into a weighted bipartite graph minimum full matching problem based on the Hungarian matching algorithm, and the association relationship between targets can be represented by a two-dimensional cost matrix. For example, the target state is predicted through a constant turning rate and speed model, and the target state prediction result and the current detection result are associated and matched based on a metric function of multi-dimensional feature fusion. The optimal association matching between targets is achieved through Hungarian matching, and the state of the successfully matched target is updated through the extended Kalman filter algorithm, and then the state information of the tracked target is output. However, this method requires the manual construction of multiple metric functions, metric function weights, and complex association processes, resulting in low versatility of the method. In addition, based on this method, parameters need to be adjusted for different sensors separately, increasing development costs.

[0045] In this embodiment, a first state information sample of a first object sample collected by the vehicle at time T and a second state information sample of at least one second object sample collected by the vehicle at a historical moment before time T are obtained, and a target association matrix is ​​obtained according to the first state information sample, the second state information sample and the object association model, and an object association result sample is obtained according to the target association matrix sample, and the object association model is trained using the object association result sample to obtain a trained object association model; the object association result sample includes the first object sample and the target second object sample associated with the first object sample. The method of this embodiment can improve the prediction performance of the object association model by associating and fusing object samples at multiple moments. In addition, this embodiment does not need to train the object association model separately for sensors with different layouts using the state information collected by them, but can associate and fuse the state information of sensors with different installation layouts under the same object association model. Furthermore, this embodiment reduces the operations of logic setting, threshold setting, adjustment and screening of different sensors by manual methods, and can associate the tracking targets of all sensors through a shared object association model, thereby reducing development costs.

[0046] Reference Figure 3 , the method may include the following steps:

[0047] Step 201: Obtain a first state information sample of a first object sample collected by a vehicle at time T.

[0048] The method of this step has been described in the aforementioned step 101 and will not be repeated here.

[0049] For example, the vehicle has at least one sensor for collecting status information, such as a visual sensor, a millimeter wave radar sensor, or a laser radar.

[0050] Step 202: Acquire the initial second state information collected by each sensor at a preset time before time T.

[0051] There is at least one type of initial second state information, for example, state information such as object position, target speed, target acceleration, target frame size, heading angle, target category, sensor category of a sensor for collecting state information, etc.

[0052] Step 203 : For each type of initial second state information, weighted fusion processing is performed on the initial second state information collected by each sensor to obtain target second state information.

[0053] For example, the second state information includes the object position, and the sensor includes a visual sensor, a millimeter wave radar sensor, and a laser radar. The object position collected by the visual sensor, the object position collected by the millimeter wave radar sensor, and the object position collected by the laser radar are weighted and fused to obtain the target position.

[0054] Step 204: construct a second state information sample according to the target second state information.

[0055] A set of all types of target second state information is determined as a second state information sample. In other words, the second state information sample includes all types of target second state information.

[0056] By acquiring the initial second state information of multiple sensors, weighted fusion is performed on the initial second state information of the same type, and the second state sample information is constructed according to the target second state information obtained by weighted fusion. The second state sample information thus obtained is a fusion of the initial second state information of multiple sensors, and has higher accuracy. In addition, subsequent object association model training is performed based on the fused second state information samples, rather than training the object association model for each type of sensor separately, which can save model training costs, and the object association model thus obtained has higher applicability and can be used to perform object association on objects collected by sensors of different types.

[0057] Step 205: Acquire a first state feature of a first state information sample.

[0058] For example, the first state feature of the first state information sample can be extracted by a feature extraction module in the object association model. The first state feature reflects all types of target second state information in the first state information sample.

[0059] Step 206: for each preset moment, obtain the second state feature of the second state information sample collected at the preset moment.

[0060] For example, for each preset moment, the second state feature of the second state information sample is extracted by the feature extraction module in the object association model. Figure 4 There are three preset moments, namely, moment t1, moment t2 and moment t3. For these three preset moments, the second state features of the second state information samples at each preset moment are extracted respectively.

[0061] Step 207, obtaining a target association matrix corresponding to the preset time according to the first state feature, the second state feature of the second state information sample collected at the preset time, and the object association model.

[0062] In the case where there is at least one first object sample, one first object sample has one corresponding first state information sample;.

[0063] For example, step 207 may include the following sub-steps:

[0064] Sub-step A1: for each first state information sample of the first object sample, combine the first state feature of the first state information sample and the second state feature of the second state information sample to obtain at least one state feature pair.

[0065] A state feature pair includes a first state feature and a second state feature.

[0066] For example, refer to Figure 2 There are n first object samples, and each first object sample has 13 types of first state information samples, including: target location Target speed Target acceleration Target box size (l s ,w s ), heading angle θ s , target category c s , sensor categories s , sensor identification, and the time difference between time T and a preset time of the second state information sample to be analyzed.

[0067] Figure 4 The first feature of the first state information sample of the first object at time T (time t4) is shown, and the first feature is marked as the sensor target feature. For the first first object sample, feature extraction is performed on its first state information sample to obtain the first state feature sample of the first first object sample. for: in, and Respectively represent the first state feature of the first state information sample of the first first object sample, the first state feature of the second state information sample of the first first object sample, and the first state feature of the 13th state information sample of the first first object sample. The first state feature of the nth first object for: in, and They respectively represent the first state feature of the 1st state information sample of the nth first object sample, the first state feature of the 2nd state information sample of the nth first object sample, and the first state feature of the 13th state information sample of the nth first object sample.

[0068] This embodiment obtains the second feature of the second state information sample of the second object at three preset times t1, t2, and t3 before time t4, and marks the second feature as a multi-sensor fusion target feature. They respectively represent the second feature of the first second object, the second feature of the second object, and the second feature of the m1th second object at time t1. They respectively represent the second feature of the first second object, the second feature of the second object, and the second feature of the m2th second object at time t2. They respectively represent the second feature of the first second object, the second feature of the second object, and the second feature of the m3th second object at time t3.

[0069] Reference Figure 4 , there are n first object samples and m1 second object samples at time t1, then the first features of all first object samples are traversed, and for each first state feature, it is combined one by one with the second feature of the second object sample to obtain at least one state feature pair.

[0070] Further, in Figure 4 There are n first object samples in total, then the first state features of the n first object samples are respectively combined with the second state features of each second state information sample to obtain at least one state feature pair.

[0071] For example, for the first state feature of the first object sample Combining it with the second state feature sample of each second state information collected at time t1, the following feature pair is obtained:

[0072] Sub-step A2, embedding the sensor identification of the source sensor and the time difference between the time T and the preset time in the state feature pair to obtain the target state feature pair.

[0073] The source sensor is a sensor used to collect first state information samples.

[0074] The sensor identifier may be a sensor number, a sensor name, or other identity document (ID) of the sensor.

[0075] For example, the first feature of the first state information sample is The second feature of the second state information sample is: Then a state information feature pair sample F consisting of a first state information sample and a second state information sample i =[s,f i](i=1,2,...,t). Where s represents the first state information sample, f i represents the second state information sample at the i-th preset moment. Then, in the target feature pair F i Sensor target data in In the source sensor ID and time difference are embedded (i=1,2,...,t) these two dimensional features, so that the sensor target data is updated to Among them, T s Represents the timestamp at time T, Indicates the timestamp of the i-th preset moment.

[0076] Sub-step A3, obtaining a target association matrix according to the target state feature pair and the object association model.

[0077] For example, the target feature pair is input into the object association model, such as Figure 2 The target association matrix acquisition module shown in the figure obtains the target association matrix. Further, the first feature in the target state feature pair is used as the sensor target feature, and the second feature in the target state feature pair is used as the multi-sensor fusion target feature. The two features are respectively subjected to feature dimension increase processing, and then matrix multiplication operation is performed to obtain a high-dimensional association weight matrix, and then the high-dimensional association weight matrix is ​​subjected to feature dimension reduction processing to obtain the target association matrix.

[0078] The sensor ID of the source sensor is embedded in the state feature pair, and the object association model is trained based on the state feature pair embedded with the sensor ID. During the training process, the object association model can distinguish the state information samples collected by different sensors. For multiple sensors with different layouts, the same set of object association models can be used for model training, and after obtaining the trained object association model, the same trained object association model can be used for multiple sensors with different layouts to perform object association prediction.

[0079] For example, step 207 may include the following sub-steps:

[0080] Sub-step B1, performing feature dimension upscaling processing on the first state feature of the first state information sample to obtain a first dimension upscaling feature.

[0081] For example, refer to Figure 2 , extract the features of the first state information sample, obtain the sensor target features, input them into the feature dimension upgrading module, and obtain one-dimensional dimension upgrading features.

[0082] Sub-step B2, performing feature dimension-upgrading processing on the second state feature of the second state information sample to obtain a second dimension-upgraded feature;

[0083] For example, refer to Figure 2 , extract the features of the second state information sample, obtain the sensor target features, input them into the feature dimension upgrading module, and obtain the two-dimensional dimension upgrading features.

[0084] Sub-step B3, obtaining an association weight matrix according to the first up-dimensional feature, the second up-dimensional feature, and the matrix multiplication operation mechanism of the object association model;

[0085] For example, refer to Figure 2 , perform matrix multiplication on the first dimension-upgraded feature and the second dimension-upgraded feature to obtain a high-dimensional association weight matrix.

[0086] Sub-step B4, performing feature dimensionality reduction processing on the association weight matrix to obtain a target association matrix.

[0087] Further, see Figure 2 , two dimensionality-raising networks are used to increase the dimension of the sensor target features of the first state information sample and the multi-sensor fusion target features of the second state information sample, respectively, to obtain the first dimensionality-raising features of the sensor target data And the second dimension feature of multi-sensor fusion target data Among them, n s and n f They respectively represent the target quantity output by the sensor used to collect the first state information sample and the sensor used to collect the second state information sample, and dim represents the dimension of the feature.

[0088] Furthermore, the sensor target data and multi-sensor fusion target data Perform matrix multiplication to obtain a high-dimensional association weight matrix The specific expression is as follows:

[0089]

[0090] Among them, reshape1(·) is used to transform two-dimensional data into one-dimensional data, and its processing logic is: D1×D2→(D1×D2); reshape2(·) is used to reduce the dimension of features, and its processing logic is: D1×D2×D3×D4→D1×D3×(D2×D4). Among them, D1, D2, D3 and D4 represent the features to be processed respectively.

[0091] Reference Figure 2 After obtaining the high-dimensional correlation weight matrix, the high-dimensional correlation weight matrix is ​​normalized (for example, LayerNorm normalization) and data dimension reduction is performed to obtain the processed correlation matrix Then, the Softmax function is used to process the first dimension of the association matrix to obtain the target association matrix.

[0092] Furthermore, the dimension-raising processing logic of the two dimension-raising networks is as follows:

[0093]

[0094] Among them, L(·) represents the fully connected layer, LN(·) represents the LayerNorm layer, ReLU activation function. UpLayer means up-dimensioning the data through the fully connected layer, and Up means re-mapping the up-dimensioned data through the fully connected layer.

[0095] The processing logic for reducing the dimension of the association weight matrix is ​​as follows:

[0096]

[0097] Among them, the fully connected layer in DownLayer is used to reduce the dimension of the data, and the fully connected layer in Down is used to rearrange and map the data after dimensionality reduction.

[0098] Step 208: for each preset moment, according to the target association matrix corresponding to the preset moment, obtain the initial second object sample with the greatest similarity to the first object sample.

[0099] For example, refer to Figure 4 , for the preset time t1, according to the second feature of the second state information sample collected at the preset time t1, and the first feature of the first state information collected at time T (time t4), the target association matrix obtained is:

[0100]

[0101] Among them, p 11 represents the similarity between the first state feature of the first first object sample and the second state feature of the first second object sample, p 1n represents the similarity between the first state feature of the nth first object sample and the second state feature of the first second object sample. m11 represents the similarity between the first state feature of the first object sample and the second state feature of the m1th second object sample, p m1n Represents the similarity between the first state feature of the nth first object sample and the second state feature of the m1th second object sample.

[0102] Reference Figure 4, for the preset time t2 and t3, the above method is also used to obtain the corresponding target association matrix. Among them, p m21 represents the similarity between the first state feature of the first object sample and the second state feature of the m2th second object sample at time t2, p m2n represents the similarity between the first state feature of the nth first object sample and the second state feature of the m2th second object sample. m31 represents the similarity between the first state feature of the first object sample and the second state feature of the m3th second object sample at time t3, p m3n Represents the similarity between the first state feature of the nth first object sample and the second state feature of the m3th second object sample.

[0103] For each column in the target association matrix, the maximum similarity value is obtained, and the second object sample corresponding to the maximum similarity value is determined as the initial second object sample with the maximum similarity to the first object sample corresponding to the column. Figure 4 , the first column of the target association matrix represents the similarity between the first first object sample and each second object sample. If p is determined from the first column 21 maximum, it means that the initial second object sample with the greatest similarity to the first first object is the second object sample.

[0104] Step 209: construct a sample set according to at least one initial second object sample.

[0105] The sample set includes all initial second object samples acquired according to the aforementioned steps.

[0106] Step 210 , obtaining the number of occurrences of each initial second object sample in the sample set, and determining the initial second object sample with the largest number of occurrences in the sample set as the target second object sample with the greatest similarity to the first object sample.

[0107] For example, refer to Figure 4 , for the preset time t1, determine p from the first column of the target association matrix at the preset time 21 maximum, it means that the initial second object sample with the largest similarity to the first first object is the second second object sample; for the preset time t2, from the first column of the target association matrix at the preset time, determine p 21 maximum, it means that the initial second object sample with the largest similarity to the first first object is the second second object sample; for the preset time t3, from the first column of the target association matrix at the preset time, determine p 31The maximum indicates that the initial second object sample with the greatest similarity to the first first object sample is the third second object sample. Then the sample set includes: the second second object sample, the second second object sample, and the third second object sample. In the sample set, the second second object sample appears the most times, so the target second object sample with the greatest similarity to the first first object sample is the second second object sample.

[0108] For example, the target association matrix can be expressed as The target association matrix is ​​used to fuse associated objects. Specifically, the maximum element position in each column of the target association matrix is ​​assigned a value of 1, and the remaining positions are assigned a value of 0. The second object sample corresponding to the position assigned a value of 1 is the initial second object sample corresponding to the first object sample. Based on the method of the embodiment, a first object sample can only determine one target second object sample, that is, each sensor target can only be associated with one multi-sensor fusion target.

[0109] Step 211: determine the first state information sample and the second state information sample as model training samples, and determine the object association result as the model training label.

[0110] The object association result can be expressed in the form of a target association pair, including a first object sample and a target second object sample with the greatest similarity to the first object sample. The object association result is a determined true value, which is used as a model training label.

[0111] For example, traverse all target association matrices and convert the target association matrix into an association pair set according to the sensor target ID and the multi-sensor fusion target ID. Among them, ID s a first state information sample representing a first object sample, The second state information sample of the first second object, the second state information sample of the second second object, and the second state information sample of the jth second object are respectively represented. The association pair set includes the information in the sample set. Among them, the sensor target ID represents the ID of the first object sample, for example, it can be the number of the first object sample, and the multi-sensor fusion target ID represents the ID of the second object sample, for example, it can be the number of the second object sample.

[0112] Step 212: Use the model training samples and the model training labels to train the object association model to obtain a trained object association model.

[0113] For example, by constructing a loss function for the target association matrix, the object association model is trained according to the loss function. Among them, the object association model can be used to associate objects of multiple sensors, and the object association model can be represented as a multi-sensor target association network model. The optimal network weights of the object association model are obtained through training, and the optimal network weights obtained through training are provided to the network model for reasoning. For example, the loss L of the model training can be constructed by classification Focal loss or by binary cross entropy.

[0114] In this embodiment, the original sensor tracking data (i.e., the first state information sample) and the multi-sensor fusion target data (i.e., the second state information sample) corresponding to each moment are collected and acquired, and the target association between the two data is annotated according to the target feature pair, so as to generate the true value required by the object association model in the supervised training process, and the true value is the label used to train the object association model. For example, referring to Figure 4 , respectively obtain the second state information samples of the second object samples collected by the vehicle through the sensor at the three preset moments t1, t2 and t3, and extract the features of the second state information samples to obtain Figure 4 The multi-sensor fusion target feature shown. Further, if the vehicle has multiple sensors, then at each preset time, the state information collected by each sensor is obtained respectively, and the same type of state information collected by multiple sensors is weightedly fused to obtain a second state information sample.

[0115] The following takes the multi-sensor fusion target feature at time t1 as an example. Figure 4 The multi-sensor fusion target feature in is exemplified as follows: at time t1, there are m1 samples of the second object collected, then the multi-sensor fusion target feature at time t1 is the second state information sample of the first second object sample The second state information sample of the second second object sample To the second state information sample of the m1th second object sample Each second state information sample includes data of 11 dimensions. For example, the second state information sample of the first second object sample is The 11 dimensions of data included are:

[0116]

[0117] Obtain the first state information sample of the first object sample collected by the vehicle sensor at time t4 (i.e., time T in the above embodiment), and extract the features of the first state information sample to obtain Figure 4The sensor target feature shown in FIG. Wherein, there are n first object samples collected at time t4, and the sensor target feature includes the first state information sample of the first first object sample The first state information sample of the second first object sample To the first state information sample of the nth first object sample Each second state information sample includes data of 13 dimensions. For example, for the first state information sample of the first first object sample, The 13 dimensions of data included are:

[0118] For the second state information sample at time t1, the corresponding multi-sensor fusion target feature and the sensor target feature of the first state information sample at time t4 are associated to obtain a target association matrix. The elements in the target association matrix represent the similarity between the object samples corresponding to the two state information samples. Figure 4 , in the incidence matrix, p m1n Represents the similarity between the m1th second object sample at time t1 and the nth first object sample at time t4.

[0119] After obtaining the target association matrix between the second state information sample at each preset moment and the first state information sample at moment t4, multiple target association matrices are fused with multiple timestamp target association matrices to obtain target association pairs. The target association pair is an association pair set including multiple association pairs. Specifically, for each first state information sample, the second state information sample with the greatest association degree is obtained, and a sample set corresponding to the first state information sample is constructed.

[0120] Further, for the first state information sample of the first object sample Statistics and their corresponding second status information The second state information sample with the largest number of occurrences is used to determine the second object sample corresponding to it as the target second object sample of the first object sample. i Includes multiple values, such as k i It can be expressed as:

[0121]

[0122] in, and They respectively represent the second state information sample of the 1st second object sample at time t1, and the second state information sample of the m1th second object sample at time t1. They respectively represent the second state information sample of the 1st second object sample at time t2 and the second state information sample of the m2th second object sample at time t2. They respectively represent the second state information sample of the 1st second object sample at time t3 and the second state information sample of the m3th second object sample at time t3.

[0123] The first object sample and the target second object sample corresponding to the first object sample can be represented in the form of a target association pair. Figure 4 , for the first feature of the first object sample at time t4 The corresponding target association pair can be expressed as The second feature of the target second object sample corresponding to the first first object sample. The first feature of the second first object sample at time t4 The corresponding target association pair can be expressed as The first feature of the nth first object sample at time t4 is represented by the second feature of the target second object sample corresponding to the second first object sample. The corresponding target association pair can be expressed as A second feature of a target second object sample corresponding to the nth first object sample.

[0124] The sensor's target tracking perception algorithm can be used to track objects of different categories and output the target object's ID, location, and motion information. Specifically, the target tracking task is first performed on the original perception data of each sensor, and then the target tracking results output by all sensors are associated with the target, and finally the tracking results of multiple sensors are fused. By independently developing the two tasks of sensor tracking algorithm and target association method, the purpose of effectively reducing development difficulty and cost can be achieved. Based on the method of this embodiment, multi-sensor target association can be achieved quickly, with a high success rate and easy maintenance.

[0125] By extracting the same type of features from the state information samples of objects tracked by different sensors, sensor objects with different installation layouts can be associated and fused under the same network framework (object association model) to reduce the manual operation steps of logic setting, threshold setting, adjustment and screening of different sensors. At the same time, by associating and fusing targets at multiple times, the performance of the network model can be improved. In addition, the object association model obtained based on the method of this embodiment can be reused in the installation and configuration scenarios of a variety of different sensors without retraining the object association model, thereby greatly reducing the training cost.

[0126] Reference Figure 5 , the object association method may include the following steps:

[0127] Step 301 : acquiring first state information of a first object collected by a vehicle at a current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment.

[0128] The second object and the second state information correspond one to one.

[0129] The method of this step can refer to the description of the aforementioned step 101, which will not be repeated here.

[0130] Step 302: Input the first state information sample and the second state information sample into the trained object association model to obtain a target second object associated with the first object.

[0131] The trained object association model is obtained through the object association model training method in the aforementioned embodiment.

[0132] Acquire the first state information of the first object collected by the vehicle at the current moment, and the second state information of at least one historical object collected by the vehicle at a historical moment before the current moment, and input the first state information and the second state information into the object association model trained according to the method of the aforementioned embodiment, so as to quickly obtain the target second object that is accurately associated with the first object.

[0133] Figure 6 Another object association model training method provided in the present application embodiment is referred to Figure 6 The method may include the following steps: Step S1, obtaining target data of multiple sensors at time T.

[0134] Specifically, the original target tracking data acquired by all sensors of the vehicle at time T is obtained, and the following 11 types of target data are extracted from each original target data: object position, target speed, target acceleration, target frame size, heading angle, target category, and sensor category. The target data is equivalent to the first state information sample in the above embodiment.

[0135] Specifically, get the timestamp as T s All sensor target raw tracking data at the time Where n s Output the number of targets for the sensor. Extract the following 11 types of data from each raw target data: object position Target speed Target acceleration Target box size (l s ,w s ), heading angle θ s , target category c s , sensor categoriess .

[0136] Step S2, acquiring fused target data of multiple sensors at at least one preset time before time T.

[0137] The timestamp of the sensor target data is obtained, and the multi-sensor original associated target data of multiple consecutive timestamps closest to the timestamp are obtained. The following 11 types of fused target data are extracted from each original associated target data: object position, target speed, target acceleration, target frame size, heading angle, target category, sensor category. The fused target data is equivalent to the second state information sample in the aforementioned embodiment.

[0138] Get distance T s The most recent t consecutive timestamps are Multi-sensor raw associated target data Where n f Output the number of targets for target association. Extract the following 11 types of fused target data from each original associated target data: object position Target speed Target acceleration Target box size (l f ,w f ), heading angle θ f , target category c f , sensor categories f . T s The time is equivalent to the time T in the aforementioned embodiment.

[0139] Step S3, generating an initial feature pair according to the target data and the fused target data, and inserting the source sensor ID and the timestamp difference into the initial feature pair to obtain the target feature pair.

[0140] Specifically, a target feature pair is generated according to the sensor target data and the multi-sensor fusion target data, and a source sensor ID and a timestamp difference are embedded in the sensor target data in the target feature pair.

[0141] Based on sensor target data and multi-sensor fusion target data Generate target feature pair F i =[s,f i ](i=1,2,...,t). Then for the target feature pair F i Sensor target data in Embed source sensor ID and timestamp difference Update the sensor target data s to

[0142] Step S4, traverse each set of data in the target feature pair, input the target data and fused target data in the target feature pair into the target association network model, and obtain the target association matrix.

[0143] For the obtained target feature pair F i =[s,f i ](i=1,2,...,t), traverse all the data in the target feature pair, and convert the sensor target data in the feature pair and multi-sensor fusion target data Input target association network model, output target association matrix

[0144] Step S5, fusing the associated targets according to the target association matrix to obtain a fusion result.

[0145] According to the target association matrix, the associated targets are fused, wherein the associated targets are equivalent to the first object sample in the above embodiment and the target second object sample with the greatest similarity to the first object sample.

[0146] Specifically, for the target association matrix, only the maximum element value of each column in the association matrix is ​​taken, that is, each sensor target can only be associated with one multi-sensor fusion target. Traverse the association matrix at all preset times, convert the target association matrix into a target association pair according to the sensor target ID and the multi-sensor fusion target ID, and then take the multi-sensor fusion target with the most association times for each sensor target in all association pairs as the association success target result.

[0147] Step S6, training a supervised multi-sensor target association network according to the fusion result to obtain a trained multi-sensor target association network.

[0148] The original sensor tracking data and multi-sensor fusion target data corresponding to each moment are collected and acquired, and the target association between the two state information data is labeled according to the target characteristics to obtain the true value required for the supervised training of the network model, and use it as the label of the training model; by constructing a loss function for the target association matrix, the multi-sensor target association network model training is completed, and the optimal network weights obtained through training are provided to the network model for reasoning.

[0149] Based on the method of this embodiment, the evaluation of object similarity based on machine learning is realized, and the tracking targets of all sensors can be associated through a shared network model (object association model), thereby minimizing the development cost and use cost. In this embodiment, the same type of features can be extracted for the tracking target data of different sensors, and then the sensor objects with different installation layouts can be associated and fused under the same network framework to reduce the manual operations such as logic setting, threshold setting, adjustment and screening for different sensors. At the same time, by associating and fusing multi-timestamp objects, and training the object association model based on the fusion results, the performance of the object association model can be improved. In addition, this embodiment can be reused in the installation and configuration scenarios of sensors in various different situations without retraining the model, thereby greatly reducing the training cost.

[0150] Reference Figure 7 , the object association model training method may include the following steps:

[0151] Step F1, obtaining sensor target data, including: position, velocity, acceleration, target frame size, heading angle and category of a first object sample.

[0152] The method of this step has been described in the aforementioned step S1 and will not be repeated here.

[0153] Step F2, obtaining multi-sensor fusion target data, including: position, velocity, acceleration, target frame size, heading angle and category of the second object sample.

[0154] The method of this step has been described in the aforementioned step S2 and will not be repeated here.

[0155] Step F3, generating target feature pairs.

[0156] The method of this step has been described in the aforementioned step S3 and will not be repeated here.

[0157] Step F4, embed the sensor ID and the time difference into the target feature pair.

[0158] The sensor ID is an identifier of a sensor used to collect the first state information sample of the first object sample.

[0159] Step F5, performing multi-sensor target object association through the object association model.

[0160] The object association model in this step is the target association network model in step S4. The method of this step can refer to the description of step S4 and will not be repeated here.

[0161] Step F6, fusing the target correlation matrices at multiple moments.

[0162] The method of this step has been described in the aforementioned step S5 and will not be repeated here.

[0163] Step F7, obtaining target association pairs.

[0164] The method of this step has been described in the aforementioned step S5 and will not be repeated here.

[0165] Step F8, obtaining sensor target and multi-sensor fusion target annotation information, using it as a label to train the object association model, and obtaining a trained object association model.

[0166] The method of this step has been described in the aforementioned step S6 and will not be repeated here.

[0167] The target association algorithm in the related art requires manual processing and feature construction of target data, and requires object matching based on a complex association matching algorithm. In this embodiment, by extracting the same type of features from target data tracked by different sensors, sensor targets with different installation layouts are associated and fused under the same network framework, and by associating and fusing multi-timestamp targets, an object association model is trained based on the fusion results. This embodiment provides a target association algorithm that can adapt to multiple sensors at the same time and is simple to develop and maintain, which minimizes the development and use costs and improves the robustness and prediction accuracy of the object association model.

[0168] refer to Figure 8 , an object association model training device provided by an embodiment of the present application, the device 40 includes: a first acquisition module 401, used to obtain a first state information sample of a first object sample collected by a vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before time T; the second object sample and the second state information sample correspond one to one; a second acquisition module 402, used to obtain a target association matrix according to the first state information sample, the second state information sample and the object association model; the target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample; a third acquisition module 403, used to obtain an object association result sample according to the target association matrix, and train the object association model based on the object association result sample to obtain a trained object association model; the object association result sample includes a first object sample, and a target second object sample with the greatest similarity to the first object sample.

[0169] Optionally, the vehicle has at least one sensor for collecting state information; the first acquisition module 401 includes: a first acquisition submodule, used to obtain the initial second state information collected by each sensor at a preset time before time T; there is at least one type of initial second state information; a second acquisition submodule, used to perform weighted fusion processing on the initial second state information collected by each sensor for each type of initial second state information to obtain target second state information; a first construction submodule, used to construct a second state information sample based on the target second state information.

[0170] Optionally, in the case where there is at least one preset moment, the second acquisition module 402 includes: a third acquisition submodule, used to acquire the first state characteristics of the first state information sample; a fourth acquisition submodule, used to acquire, for each preset moment, the second state characteristics of the second state information sample collected at the preset moment; and a fifth acquisition submodule, used to obtain a target association matrix corresponding to the preset moment based on the first state characteristics, the second state characteristics of the second state information sample collected at the preset moment, and the object association model.

[0171] Optionally, in the case where there is at least one preset moment, the third acquisition module 403 includes: a sixth acquisition submodule, used to obtain, for each preset moment, an initial second object sample with the greatest similarity to the first object sample according to a target association matrix corresponding to the preset moment; a second construction submodule, used to construct a sample set based on at least one initial second object sample; and a seventh acquisition submodule, used to obtain the number of times each initial second object sample appears in the sample set, and determine the initial second object sample with the greatest number of appearances in the sample set as the target second object sample with the greatest similarity to the first object sample.

[0172] Optionally, the third acquisition module 403 includes: a first determination submodule, used to determine the first state information sample and the second state information sample as model training samples, and determine the object association result as a model training label; an eighth acquisition submodule, used to train the object association model using the model training samples and the model training labels to obtain a trained object association model.

[0173] Optionally, when there is at least one first object sample, one first object sample has a corresponding first state information sample; the second acquisition module 402 includes: a ninth acquisition submodule, for combining, for each first state information sample of the first object sample, the first state feature of the first state information sample and the second state feature of the second state information sample to obtain at least one state feature pair; a tenth acquisition submodule, for embedding the sensor identifier of the source sensor and the time difference between the T moment and the preset moment in the state feature pair to obtain a target state feature pair; the source sensor is a sensor for collecting the first state information sample; an eleventh acquisition submodule, for obtaining a target association matrix based on the target state feature pair and the object association model.

[0174] Optionally, the second acquisition module 402 includes: a twelfth acquisition sub-module, which is used to perform feature dimensionality upgrading processing on the first state features of the first state information sample to obtain the first upgraded dimensionality features; a thirteenth acquisition sub-module, which is used to perform feature dimensionality upgrading processing on the second state features of the second state information sample to obtain the second upgraded dimensionality features; a fourteenth acquisition sub-module, which is used to obtain the association weight matrix based on the first upgraded dimensionality features, the second upgraded dimensionality features, and the matrix multiplication operation mechanism of the object association model; and a fifteenth acquisition sub-module, which is used to perform feature dimensionality reduction processing on the association weight matrix to obtain the target association matrix.

[0175] In this embodiment, the prediction performance of the object association model can be improved by associating and fusing object samples at multiple moments. In addition, this embodiment does not need to train object association models separately for sensors with different layouts using their collected status information. Instead, the status information of sensors with different installation layouts can be associated and fused under the same object association model. Furthermore, this embodiment reduces the need to manually perform logic setting, threshold setting, adjustment, and screening operations on different sensors, and can associate the tracking targets of all sensors through a shared object association model, thereby reducing development costs. Fig. 9 , an embodiment of the present application provides an object association device, device 50: a fourth acquisition module 502, used to obtain the first state information of the first object collected by the vehicle at the current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment; the second object and the second state information correspond one to one; a fifth acquisition module 503, used to input the first state information sample and the second state information sample into a trained object association model to obtain a target second object associated with the first object; wherein the trained object association model is obtained by the object association model training method of any one of the above embodiments.

[0176] In this embodiment, first state information of a first object collected by the vehicle at the current moment and second state information of at least one historical object collected by the vehicle at a historical moment before the current moment are obtained, and the first state information and the second state information are input into an object association model trained according to the method of the aforementioned embodiment, so that a target second object accurately associated with the first object can be quickly obtained.

[0177] Fig.10 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0178] Reference Fig.10 , the electronic device 600 may include one or more of the following components: a processing component 602, a memory 606, a power component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 616, and a communication component 616. The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602. The memory 606 is used to store various types of data to support operations on the electronic device 600. Examples of these data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, multimedia, etc. Memory 606 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0179] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0180] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0181] The audio component 610 is used to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0182] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0183] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0184] The communication component 616 is used to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 6G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0185] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement an object association model training method or an object association method provided in an embodiment of the present application. In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions may be executed by a processor 620 of the electronic device 600 to complete the above method. For example, the non-temporary storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0186] Fig.11 is a block diagram of an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. Fig.11, the electronic device 700 includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions that can be executed by the processing component 722, such as an application. The application stored in the memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to execute an object association model training method or an object association method provided in an embodiment of the present application. The electronic device 700 may also include a power supply component 726 configured to perform power management of the electronic device 700, a wired or wireless network interface 770 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 778. The electronic device 700 can operate based on an operating system stored in the memory 732, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0187] An embodiment of the present application discloses a vehicle, including a training device for an object association model, an object association device, or an electronic device according to the aforementioned embodiment.

[0188] An embodiment of the present application also provides a computer program product, including a computer program, which implements an object association model training method or an object association method when the computer program is executed by a processor.

[0189] An embodiment of the present application also provides a vehicle, comprising a training device for an object association model as in the aforementioned embodiment, an object association device as in the aforementioned embodiment, or an electronic device as in the aforementioned embodiment.

[0190] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0191] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for training an object association model, characterized in that: include: Acquire a first state information sample of a first object sample collected by a vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before the time T; the second object sample and the second state information sample correspond one to one; Obtaining a target association matrix according to the first state information sample, the second state information sample and an object association model; The target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample; Obtain object association result samples according to the target association matrix, and train the object association model based on the object association result samples to obtain a trained object association model; the object association result samples include the first object sample and the target second object sample with the greatest similarity to the first object sample.

2. The method according to claim 1, characterized in that The vehicle has at least one sensor for collecting state information; obtaining a second state information sample of at least one second object sample collected by the vehicle at a preset time before the T time, including: Acquire initial second state information collected by each of the sensors at a preset time before time T; the initial second state information has at least one type; For each type of the initial second state information, weighted fusion processing is performed on the initial second state information collected by each sensor to obtain target second state information; The second state information sample is constructed according to the target second state information.

3. The method according to claim 1, characterized in that: In the case where there is at least one of the preset moments, obtaining a target association matrix according to the first state information sample, the second state information sample and the object association model includes: Acquire a first state feature of the first state information sample; For each of the preset moments, obtaining a second state feature of a second state information sample collected at the preset moment; The target association matrix corresponding to the preset moment is obtained according to the first state feature, the second state feature of the second state information sample collected at the preset moment, and the object association model.

4. The method according to claim 3, characterized in that In the case where there is at least one of the preset moments, obtaining the object association result according to the target association matrix includes: For each of the preset moments, obtaining an initial second object sample having the greatest similarity to the first object sample according to the target association matrix corresponding to the preset moment; constructing a sample set based on at least one initial second object sample; The number of times each of the initial second object samples appears in the sample set is obtained, and the initial second object sample with the largest number of appearances in the sample set is determined as the target second object sample with the greatest similarity to the first object sample.

5. The method according to claim 1, characterized in that Training the object association model based on the object association result sample to obtain a trained object association model includes: Determine the first state information sample and the second state information sample as model training samples, and determine the object association result as a model training label; The object association model is trained using the model training samples and the model training labels to obtain a trained object association model.

6. The method according to claim 1, characterized in that In the case where there is at least one first object sample, one first object sample has one corresponding first state information sample; According to the first state information sample, the second state information sample and the object association model, a target association matrix is ​​obtained, including: For each first state information sample of the first object sample, combining a first state feature of the first state information sample and a second state feature of the second state information sample to obtain at least one state feature pair; The sensor identifier of the source sensor and the time difference between the T moment and the preset moment are embedded in the state feature pair to obtain a target state feature pair; the source sensor is a sensor used to collect the first state information sample; According to the target state feature pairs and the object association model, the target association matrix is ​​obtained.

7. The method according to any one of claims 1 to 6, characterized in that: According to the first state information sample, the second state information sample and the object association model, a target association matrix is ​​obtained, including: Performing feature dimension-upgrading processing on the first state feature of the first state information sample to obtain a first dimension-upgraded feature; Performing feature dimension-upgrading processing on the second state feature of the second state information sample to obtain a second dimension-upgraded feature; Obtaining an association weight matrix according to the first upscaled feature, the second upscaled feature, and a matrix multiplication operation mechanism of the object association model; Perform feature dimension reduction processing on the association weight matrix to obtain the target association matrix.

8. An object association method, characterized in that: include: Acquire first state information of a first object collected by a vehicle at a current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment; the second object and the second state information correspond one to one; The first state information sample and the second state information sample are input into a trained object association model to obtain a target second object associated with the first object; wherein the trained object association model is obtained by the object association model training method described in any one of claims 1 to 7.

9. A training device for an object association model, characterized in that: include: A first acquisition module is used to acquire a first state information sample of a first object sample collected by the vehicle at time T, and a second state information sample of at least one second object sample collected by the vehicle at a preset time before the time T; the second object sample and the second state information sample correspond one to one; a second acquisition module, configured to obtain a target association matrix according to the first state information sample, the second state information sample and an object association model; the target association matrix is ​​used to reflect the similarity between the first state information sample and the second state information sample; The third acquisition module is used to obtain object association result samples according to the target association matrix, and train the object association model based on the object association result samples to obtain a trained object association model; the object association result samples include the first object sample and the target second object sample with the greatest similarity to the first object sample.

10. An object association device, characterized in that: include: a fourth acquisition module, configured to acquire first state information of a first object collected by a vehicle at a current moment, and at least one second state information of at least one historical object collected by the vehicle at a historical moment before the current moment; the second object and the second state information correspond one to one; A fifth acquisition module is used to input the first state information sample and the second state information sample into a trained object association model to obtain a target second object associated with the first object; wherein the trained object association model is obtained by the object association model training method described in any one of claims 1 to 7.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A vehicle, characterized in that: It comprises the training device of the object association model as claimed in claim 9, the object association device as claimed in claim 10, or the electronic device as claimed in claim 11.

Images