A method, device and storage medium for processing vehicle tracking data

Abstract

The present application relates to a method, device and storage medium for processing vehicle tracking data, including obtaining a target vehicle tracking image; obtaining an initial perception box corresponding to an environmental vehicle in a historical vehicle tracking image at the previous moment based on a historical vehicle trajectory; identifying the target vehicle tracking image based on a preset vehicle perception model to obtain a full-image vehicle detection box; performing pairing processing on the initial perception box to obtain an environmental vehicle pairing result; if there is a target vehicle pair, obtaining a predicted perception box based on a preset motion model and the initial perception box corresponding to the target vehicle pair; performing omission detection on the target vehicle pair based on the full-image vehicle detection box, the predicted perception box and a preset omission detection model; and determining an update strategy for the historical vehicle trajectory based on the detection result. The present application determines a target vehicle pair through historical trajectory data and a preset motion model, determines whether there is an omission of the target vehicle pair, and determines an update strategy for the historical vehicle trajectory based on the detection result, thereby improving the vehicle detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, device, and storage medium for processing vehicle tracking data. Background Art

[0002] The forward vehicle perception technology uses a deep learning model to detect vehicles. However, in actual applications, there will be a situation where the environmental vehicles in front of a vehicle block each other in the forward perception image of the vehicle, which is called the front-to-back vehicle occlusion phenomenon here. Figure 1 Summary of the Invention

[0003] In view of this, the present application proposes a method, device, and storage medium for processing vehicle tracking data. It can at least solve the problem of missed detection caused by front-to-back vehicle occlusion that may exist in the forward perception image of a vehicle on the road, improve the vehicle detection accuracy, and further improve the robustness of the vehicle tracking data processing algorithm in the field of intelligent driving.

[0004] According to one aspect of the present application, a method for processing vehicle tracking data is provided. In a possible implementation, the method includes:

[0005] Obtain a target vehicle tracking image; the target vehicle tracking image is an image obtained by the in-vehicle camera device of the current vehicle at the current moment and includes at least two environmental vehicles in front of the vehicle;

[0006] Based on at least two historical vehicle trajectories, obtain the initial perception frames corresponding to the environmental vehicles in front of the vehicle in the historical vehicle tracking image at the previous moment; the historical vehicle trajectory represents the driving trajectory corresponding to a single environmental vehicle in the historical vehicle tracking image;

[0007] Perform whole-image vehicle recognition on the target vehicle tracking image based on a preset vehicle perception model to obtain the whole-image vehicle detection frame corresponding to the target vehicle tracking image;

[0008] Perform pairing processing on the initial perception frames corresponding to each environmental vehicle based on a preset pairing model to obtain an environmental vehicle pairing result;

[0009] When the result of the environmental vehicle pairing indicates the existence of a target vehicle pair, based on a preset motion model and the initial perception frames corresponding to the target vehicle pair, predict the positions of the perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle tracking image, respectively, to obtain the predicted perception frames of the first environmental vehicle and the second environmental vehicle.

[0010] Based on the full-image vehicle detection frames, the predicted perception frames, and a preset missed detection identification model, perform missed detection for the target vehicle pair on the target vehicle tracking image.

[0011] When it is determined that there is no missed detection, update each historical vehicle trajectory based on the full-image vehicle detection frames to obtain the corresponding current vehicle trajectory.

[0012] Further, when it is determined that there is a missed detection, determine the target sub-region corresponding to the missed target vehicle pair in the target vehicle tracking image.

[0013] Based on the preset vehicle perception model, perform sub-graph target extraction on the target sub-region to obtain the sub-graph vehicle detection frames corresponding to the target sub-region.

[0014] Based on the full-image vehicle detection frames and the sub-graph vehicle detection frames, update each historical vehicle trajectory to obtain the corresponding current vehicle trajectory.

[0015] Further, the method further includes: when it is detected that there is a full-image vehicle detection frame or a sub-graph vehicle detection frame in the target vehicle tracking image that does not match the historical vehicle trajectory, create a new vehicle tracking trajectory based on the full-image vehicle detection frame or the sub-graph vehicle detection frame that does not match the historical vehicle trajectory.

[0016] In a possible implementation, determining the target sub-region corresponding to the missed target vehicle pair in the target vehicle tracking image includes obtaining the predicted perception frame corresponding to the first environmental vehicle and the predicted perception frame corresponding to the second environmental vehicle in the missed target vehicle pair.

[0017] Enlarge the predicted perception frame with a larger area among the predicted perception frames of the first environmental vehicle and the second environmental vehicle to obtain the target sub-region.

[0018] Further, the method further includes, when the result of the environmental vehicle pairing indicates the non-existence of a target vehicle pair, updating each historical vehicle trajectory based on the full-image vehicle detection frames to obtain the corresponding current vehicle trajectory.

[0019] In a possible implementation, when the environmental vehicle pairing result indicates the existence of a target vehicle pair, the method further includes:

[0020] Obtain the bottom edge position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the target vehicle pair respectively;

[0021] Determine the larger and smaller values of the bottom edge position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the vehicle body height direction;

[0022] Determine the environmental vehicle corresponding to the smaller value as the front vehicle, and the environmental vehicle corresponding to the larger value as the rear vehicle;

[0023] The front vehicle represents the environmental vehicle closest to the current vehicle in the field of view of the on-vehicle camera device, and the rear vehicle represents the environmental vehicle whose distance from the current vehicle exceeds that of the front vehicle.

[0024] In a possible implementation, the pairing process of the initial perception frames corresponding to each environmental vehicle based on a preset pairing model to obtain an environmental vehicle pairing result includes:

[0025] Obtain the position coordinates of the initial perception frames in the historical vehicle tracking image at the previous moment in the camera coordinate system;

[0026] Based on the position coordinates in the camera coordinate system, obtain the corresponding position coordinates in the vehicle body coordinate system;

[0027] Based on the position coordinates in the vehicle body coordinate system, obtain the lateral spacing between environmental vehicles pairwise; the lateral spacing is the distance between environmental vehicles pairwise along the vehicle axis direction in the vehicle body coordinate system;

[0028] Based on the lateral spacing, the intersection over union and width ratio of the two initial perception frames corresponding to the lateral spacing, obtain the environmental vehicle pairing result.

[0029] In a possible implementation, the omission detection for the target vehicle pair in the target vehicle tracking image based on the whole-image vehicle detection frame, the predicted perception frame, and a preset omission recognition model includes:

[0030] Obtain the intersection over union of the predicted perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle pair and the whole-image vehicle detection frame respectively, to obtain a first set of intersection over union corresponding to the first environmental vehicle and a second set of intersection over union corresponding to the second environmental vehicle;

[0031] Obtain the maximum value in the first set of intersection over union, and the first index information of the corresponding whole-image vehicle detection frame;

[0032] Obtain the maximum value in the second set of intersection over union (IoU), and the second index information of the whole-image vehicle detection box corresponding thereto;

[0033] If the maximum value in the first set of IoU and the maximum value in the second set of IoU are both greater than a preset threshold, and the first index information is not equal to the second index information, it is determined that there is no missed detection of the target vehicle pair; otherwise, there is a missed detection.

[0034] According to another aspect of the present application, there is provided a vehicle tracking data processing device, including:

[0035] An image acquisition module, configured to obtain a target vehicle tracking image; the target vehicle tracking image is an image including at least two environmental vehicles in front of the vehicle, acquired by the on-vehicle camera device of the current vehicle at the current moment;

[0036] A historical trajectory query module, configured to obtain, based on at least two historical vehicle trajectories, the initial perception frames corresponding to the environmental vehicles in front of the vehicle in the historical vehicle tracking image at the previous moment; the historical vehicle trajectory represents the driving trajectory corresponding to a single environmental vehicle in the historical vehicle tracking image;

[0037] An image target extraction module, configured to perform whole-image vehicle recognition on the target vehicle tracking image based on a preset vehicle perception model, to obtain the whole-image vehicle detection box corresponding to the target vehicle tracking image;

[0038] A target vehicle pair detection module, configured to perform pairing processing on the initial perception frames corresponding to each environmental vehicle based on a preset pairing model, to obtain an environmental vehicle pairing result;

[0039] A trajectory prediction module, configured to, when the environmental vehicle pairing result indicates the existence of a target vehicle pair, predict the positions of the perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle pair in the target vehicle tracking image based on a preset motion model and the initial perception frames corresponding to the target vehicle pair, respectively obtaining the predicted perception frames of the first environmental vehicle and the second environmental vehicle;

[0040] A missed detection judgment module, configured to perform missed detection of the target vehicle pair on the target vehicle tracking image based on the whole-image vehicle detection box, the predicted perception frames, and a preset missed detection recognition model;

[0041] A vehicle trajectory update module, configured to, when it is determined that there is no missed detection, update each historical vehicle trajectory based on the whole-image vehicle detection box, to obtain the corresponding current vehicle trajectory.

[0042] According to another aspect of the present application, there is provided a computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the method as described above.

[0043] The vehicle tracking data processing method, device and storage medium provided by the present application have the following beneficial effects:

[0044] Determine the target vehicle pair through historical trajectory data and a preset motion model, and perform omission detection on the target vehicle tracking image for the target vehicle pair based on the whole-image vehicle detection frame, the predicted perception frame and a preset omission detection model; determining an update strategy for the historical vehicle trajectory based on the detection result improves the vehicle detection accuracy, and thus can improve the robustness of the vehicle tracking algorithm in the field of intelligent driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings according to these drawings without creative efforts.

[0046] Figure 1 It is a diagram showing the phenomenon of front and rear vehicle occlusion.

[0047] Figure 2 It is a flowchart of a vehicle tracking data processing method provided by an embodiment of the present application.

[0048] Figure 3 It is a flowchart of another vehicle tracking data processing method provided by an embodiment of the present application.

[0049] Figure 4 It is a flowchart of yet another vehicle tracking data processing method provided by an embodiment of the present application.

[0050] Figure 5 It is a flowchart of yet another example of a vehicle tracking data processing method provided by an embodiment of the present application.

[0051] Figure 6 It is a flowchart of still another example of a vehicle tracking data processing method provided by an embodiment of the present application.

[0052] Figure 7 It is an example of applying the method of the present application to process vehicle tracking data provided by an embodiment of the present application.

[0053] Figure 8Schematic diagram of a vehicle tracking data processing device provided by an embodiment of the present application;

[0054] Figure 9 Hardware structure block diagram of a device for a vehicle tracking data processing method provided by an embodiment of the present application. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0057] In addition, for better illustration of the present application, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present application can be implemented without some specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present application.

[0058] When vehicles driving on the road appear in the front view perception image, there will be a phenomenon of front and rear vehicle occlusion. For this phenomenon, it is inevitable to have problems of vehicle missing detection and false detection when only using vehicle perception models such as YOLO to perform vehicle recognition on the collected target vehicle tracking images. This embodiment provides a vehicle tracking data processing method, which can improve the accuracy of vehicle detection, and the specific description is as follows:

[0059] Please refer to Figure 2 , Figure 2 which is a flowchart of a vehicle tracking data processing method according to an embodiment of the present application. As Figure 2 shown, the vehicle tracking data processing method includes:

[0060] S100, obtaining a target vehicle tracking image.

[0061] In the embodiments of the present application, the target vehicle tracking image is an image obtained by the on-vehicle camera device of the current vehicle at the current moment, including images of at least two environmental vehicles in front of the vehicle.

[0062] S110. Based on at least two historical vehicle trajectories, obtain the initial perception frames corresponding to the environmental vehicles in front of the vehicle in the historical vehicle tracking image at the previous moment.

[0063] In the embodiments of the present application, the historical vehicle trajectory represents the driving trajectory corresponding to a single environmental vehicle in the historical vehicle tracking image.

[0064] S120. Based on a preset vehicle perception model, perform whole-image vehicle recognition on the target vehicle tracking image to obtain the whole-image vehicle detection frame corresponding to the target vehicle tracking image.

[0065] It should be noted that the preset vehicle perception model can be a model such as YOLO, using a backbone network with a large amount of computational complexity, such as VarGNetV2, taking the complete target vehicle tracking image as the input and outputting the whole-image vehicle detection frame.

[0066] S130. Based on a preset pairing model, perform pairing processing on the initial perception frames corresponding to each environmental vehicle to obtain the environmental vehicle pairing result.

[0067] It should be noted that the target vehicle pair is two environmental vehicles corresponding to two initial perception frames that meet the above-mentioned lateral spacing, intersection over union, and width ratio, which are screened from the historical vehicle trajectories according to the preset pairing model. The target vehicle pair is used to represent two environmental vehicles that are occluded in the field of view of the on-vehicle camera device of the vehicle and meet the preset pairing model.

[0068] Please refer to Figure 5 , in a specific embodiment, this process may further include:

[0069] S1301. Obtain the position coordinates of the initial perception frame in the historical vehicle tracking image at the previous moment in the camera coordinate system.

[0070] S1302. Based on the position coordinates in the camera coordinate system, obtain the corresponding position coordinates in the vehicle body coordinate system.

[0071] In practical applications, the coordinate conversion relationship can be obtained according to the vehicle body calibration technology. The following formula (1) is the corresponding relationship between the vehicle body coordinate system P(Xw, Yw, Zw) and the camera coordinate system (Xc, Yc, Zc).

[0072] Formula (1):

[0073]

[0074] Wherein, R is a rotation matrix and t is a translation vector. Those skilled in the art can obtain them according to the actual situation of vehicle calibration.

[0075] S1303. Obtain the lateral spacing between any two environmental vehicles based on the position coordinates in the vehicle body coordinate system.

[0076] Specifically, in this embodiment, the lateral spacing is the distance between any two environmental vehicles along the axle direction in the vehicle body coordinate system. Particularly, for the convenience of calculation, as an optional setting, the center of the rear wheel axle can be used as the reference point, with the X-axis direction facing the front of the vehicle, the Y-axis horizontally to the left, and the Z-axis upward in the vehicle body height direction. Correspondingly, the point where the center of the rear wheel axle extends to the ground (z = 0) can be set as the origin of the vehicle body coordinate system (x = 0, y = 0, z = 0).

[0077] S1304. Obtain the pairing result of environmental vehicles based on the lateral spacing, the intersection-over-union ratio and the width ratio of the two initial perception frames corresponding to the lateral spacing.

[0078] Specifically, in the vehicle body coordinate system, obtain the position coordinates of the initial perception frames in the historical vehicle tracking image at the previous moment. If the spacing in the Y direction between two initial perception frames on different vehicle trajectories is greater than the preset threshold, it is determined that there is no target vehicle pair. If the lateral spacing in the Y direction between two initial perception frames on different vehicle trajectories is less than or equal to the preset threshold, calculate the intersection-over-union ratio and the width ratio of these two initial perception frames, and judge whether a target vehicle pair is formed based on the calculation results of the intersection-over-union ratio and the width ratio. For example, it can be set that when the intersection-over-union ratio is greater than 0.5 and the width ratio is greater than 0.7, it is determined that the environmental vehicles corresponding to the two initial perception frames with a lateral spacing less than or equal to the preset threshold form a target vehicle pair.

[0079] S140. When the pairing result of environmental vehicles indicates the existence of a target vehicle pair, based on a preset motion model and the initial perception frames corresponding to the target vehicle pair, predict the positions of the perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle tracking image, and obtain the predicted perception frames of the first environmental vehicle and the second environmental vehicle respectively.

[0080] In a specific embodiment, the predicted perception frame positions at the current moment can be predicted through the preset motion model of the KF filter and the initial perception frame positions of the target vehicle pair at the previous moment. The following formulas (2) and (3) are examples of a preset motion model:

[0081] Formula (2):

[0082] x′(k) = A * x(k - 1)+B * u(k)

[0083] Formula (3):

[0084] P'(k) = A * P(k - 1) * A T + Q

[0085] Where, the current moment is the k-th moment, and the previous moment is the (k - 1)-th moment.

[0086] A is the state transition matrix,

[0087]

[0088] P is the state covariance matrix. Set the initial moment as P0, and then P changes with time.

[0089]

[0090] Q is the process noise matrix, set as the identity matrix.

[0091]

[0092] Specifically, let the state quantity x = [u, v, vu, vv], where (u, v) is the image coordinates of the center point of the vehicle perception box, and (vu, vv) is the change rate of the center point. B = 0, u = 0; that is, there is no disturbance. Then, using the preset motion model of the KF filter, the predicted perception box at the current moment can be predicted as box = (u, v, w, h), where uv is the predicted center point coordinates mentioned above, and w and h are set as the width and height of the environmental vehicle corresponding to the vehicle perception box in the previous frame.

[0093] S150. Based on the whole-image vehicle detection box, the predicted perception box, and the preset missed detection recognition model, perform missed detection on the target vehicle tracking image for the target vehicle pair.

[0094] It should be noted that under normal circumstances, the target vehicle pair includes the first environmental vehicle and the second environmental vehicle. In theory, two whole-image vehicle detection boxes corresponding to the first environmental vehicle and the second environmental vehicle should be obtained through the preset vehicle perception model. However, due to the occlusion of the vehicles in the target vehicle pair, there will be missed detection in some cases. This step is used to determine whether two whole-image vehicle detection boxes corresponding to the first environmental vehicle and the second environmental vehicle are correctly recognized in the target vehicle tracking image through the whole-image vehicle detection box and the predicted perception box.

[0095] Please refer to Figure 6 , in a specific embodiment, this process may further include:

[0096] S1501. Respectively obtain the intersection over union of the predicted perception box and the whole-image vehicle detection box corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle pair, and obtain the first set of intersection over union corresponding to the first environmental vehicle and the second set of intersection over union corresponding to the second environmental vehicle.

[0097] S1502, obtain the maximum value in the first set of intersection over union (IoU), and the first index information of the full-image vehicle detection box corresponding thereto;

[0098] S1503, obtain the maximum value in the second set of intersection over union (IoU), and the second index information of the full-image vehicle detection box corresponding thereto;

[0099] S1504, if the maximum value in the first set of intersection over union (IoU) and the maximum value in the second set of intersection over union (IoU) are both greater than the preset threshold, and the first index information is not equal to the second index information, then determine that there is no missed detection of the target vehicle pair; otherwise, there is a missed detection.

[0100] In a specific embodiment, the preset threshold can be set to 0.6. When the maximum value in the first set of intersection over union (IoU) and the maximum value in the second set of intersection over union (IoU) are both greater than 0.6, confirm whether there are two different vehicle detection boxes according to the index information of the full-image vehicle detection box.

[0101] It should be noted that when there is a maximum value that meets the preset threshold, it indicates that there is a full-image vehicle detection box in the target vehicle tracking image that matches the environmental vehicle corresponding to the target vehicle pair. When there is no missed detection, there should be two different vehicle detection boxes, that is, the first index information is not equal to the second index information; otherwise, the full-image vehicle detection box corresponding to the first environmental vehicle and / or the second environmental vehicle corresponding to the target vehicle pair is missing in the target vehicle tracking image, that is, there is a missed detection.

[0102] S160, in the case of determining that there is no missed detection, update each historical vehicle trajectory based on the full-image vehicle detection box to obtain the corresponding current vehicle trajectory.

[0103] It should be noted that this step generally includes calculating the similarity cost between the predicted perception box and the full-image vehicle detection box, and matching the full-image vehicle detection box in the target vehicle tracking image with the historical vehicle tracking trajectory at the previous moment through the KM matching algorithm. In the case of successful matching, update the successfully matched full-image vehicle detection box into the corresponding historical vehicle trajectory to obtain the corresponding current vehicle trajectory.

[0104] Those skilled in the art should understand that the similarity cost calculation in the above process involves a similarity cost matrix, extracting the 256-dimensional feature vector of the full-image vehicle detection box, and calculating the cosine similarity distance, etc. The KM matching algorithm can be implemented using the KuhnMunkras matching algorithm, and this part can be implemented with reference to related technologies and will not be elaborated in the embodiments of the present application.

[0105] In this way, the target vehicle pair is determined through historical trajectory data and a preset motion model, and omission detection for the target vehicle pair is performed on the target vehicle tracking image based on the vehicle detection frames in the entire image, the predicted perception frames, and a preset omission detection recognition model; determining an update strategy for the historical vehicle trajectory based on the detection results improves the vehicle detection accuracy, and thus can improve the robustness of the vehicle tracking algorithm in the field of intelligent driving.

[0106] Please refer to Figure 3 , Figure 3 which is a flowchart of a vehicle tracking data processing method according to an embodiment of the present application. As Figure 3 shown, based on the above embodiment, the vehicle tracking data processing method of this embodiment includes:

[0107] S260, when it is determined that there is an omission, determine the target sub-region corresponding to the omitted target vehicle pair in the target vehicle tracking image;

[0108] S270, perform sub-graph target extraction on the target sub-region based on a preset vehicle perception model to obtain a sub-graph vehicle detection frame corresponding to the target sub-region;

[0109] In this embodiment, when an omission occurs, the region corresponding to the omitted target vehicle pair is set as the target sub-region. Vehicle recognition is performed on the determined target sub-region again through a preset vehicle perception model.

[0110] In an optional embodiment, S260 may further include:

[0111] S2601, obtain the predicted perception frame corresponding to the first environmental vehicle and the predicted perception frame corresponding to the second environmental vehicle in the omitted target vehicle pair;

[0112] S2602, perform an amplification process on the predicted perception frame with a larger area among the predicted perception frames of the first environmental vehicle and the second environmental vehicle to obtain the target sub-region.

[0113] In practical applications, as an example, the predicted perception frame with a larger area among the predicted perception frames of the first environmental vehicle and the second environmental vehicle can be amplified by 1.2 times as the target sub-region. Using the predicted perception frame with a larger area as the basis for amplification can improve the effectiveness of the coverage range of the target sub-region, reduce the number of calls to the vehicle perception model for vehicle recognition of the target sub-region, and improve the data processing efficiency.

[0114] Specifically, a target sub-region is determined through the predicted perception box of the target vehicle pair. Based on the vehicle perception model, secondary vehicle recognition is performed on the target sub-region, and the missed target vehicle pair is re-detected. This process can use a lightweight backbone network, such as tiny_vargnet_v2, to output the detection results of the sub-graph vehicle box, which can also effectively improve the data processing efficiency.

[0115] S280. Update each historical vehicle trajectory based on the vehicle detection box of the entire graph and the vehicle detection box of the sub-graph to obtain the corresponding current vehicle trajectory.

[0116] It should be noted that the vehicle detection box of the sub-graph is obtained after secondary vehicle recognition of the target sub-region on the target vehicle detection image, and together with the vehicle detection box of the entire graph obtained in step S120, it constitutes the output result of vehicle recognition for the target vehicle tracking image. The method of this embodiment can improve the time efficiency of vehicle tracking data processing in the vehicle tracking scenario by obtaining the target vehicle pair and its corresponding target sub-region and fusing the vehicle recognition of the entire graph and the sub-graph on the basis of the target tracking framework in the case of missed detection.

[0117] In a preferred embodiment, before step S280, it may further include:

[0118] Calculate the intersection over union of the obtained vehicle detection box of the sub-graph and the vehicle detection box of the entire graph one by one;

[0119] Delete the vehicle detection box of the sub-graph corresponding to the intersection over union in the case where the intersection over union is higher than the preset overlap threshold.

[0120] It should be noted that both the vehicle detection box of the sub-graph and the vehicle detection box of the entire graph are the detection boxes of the environmental vehicles in the target vehicle tracking picture. There may be vehicle boxes similar to the entire graph model in the obtained vehicle detection box of the sub-graph. This step is used to identify the situation where there are both the vehicle detection box of the entire graph and the vehicle detection box of the sub-graph for the same environmental vehicle, and determine whether to delete the corresponding vehicle detection box of the sub-graph based on the overlap threshold. The above processing steps are used to identify and remove the vehicle detection box of the sub-graph similar to the detection box of the entire graph model.

[0121] In a specific embodiment, the preset overlap threshold can be set to 0.8, that is, when the intersection over union of the vehicle detection box of the sub-graph and the vehicle detection box of the entire graph is greater than 0.8, the corresponding vehicle detection box of the sub-graph is deleted.

[0122] In an embodiment, the method further includes the following steps:

[0123] S290. When it is detected that there is a full-image vehicle detection box or a sub-image vehicle detection box in the target vehicle tracking image that does not match the historical vehicle trajectory, a new vehicle tracking trajectory is created based on the full-image vehicle detection box or the sub-image vehicle detection box that does not match the historical vehicle trajectory.

[0124] In practical applications, when a new vehicle that has not appeared in the historical trajectory appears in the target vehicle tracking image, there will be a vehicle detection box that does not match the historical vehicle trajectory. At this time, the method of this embodiment can create trajectory information for the identified new vehicle.

[0125] In one embodiment, the method further includes the following steps:

[0126] When it is detected that there is an unupdated trajectory in the historical vehicle trajectory, it is determined whether to delete the unupdated trajectory based on a preset condition.

[0127] Specifically, this step is used to handle the situation where a certain environmental vehicle in front of the vehicle leaves the field of view of the on-vehicle camera device during the vehicle driving process. For example, it can be set that when a certain environmental vehicle in front is not recognized in three consecutive frames of target vehicle detection images, the historical vehicle trajectory of this vehicle stored is deleted.

[0128] Based on the above embodiment, the vehicle tracking data processing method of this embodiment can further include:

[0129] S340. When the environmental vehicle pairing result is that there is no target vehicle pair, each historical vehicle trajectory is updated based on the full-image vehicle detection box to obtain the corresponding current vehicle trajectory.

[0130] Please refer to Figure 4 , Figure 4 which shows a flowchart of a vehicle tracking data processing method according to an embodiment of the present application. As Figure 4 shown, based on the above embodiment, the vehicle tracking data processing method includes:

[0131] S240. Respectively obtain the bottom position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the target vehicle pair;

[0132] In practical applications, this process can obtain the bottom position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the image coordinate system collected by the on-vehicle camera device. The one with a smaller bottom position coordinate in this coordinate system is closer to the current vehicle. Thus, the relative distance of the environmental vehicle relative to the current vehicle can be determined through the difference in the bottom position coordinates.

[0133] S241. Determine the larger and smaller values of the bottom position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the vehicle body height direction;

[0134] S242, determine the environmental vehicle corresponding to the smaller value as the leading vehicle and the environmental vehicle corresponding to the larger value as the trailing vehicle;

[0135] The above-mentioned leading vehicle represents the environmental vehicle closest to the current vehicle within the field of view of the on-vehicle camera device, and the above-mentioned trailing vehicle represents the environmental vehicle whose distance from the current vehicle exceeds that of the leading vehicle.

[0136] Please refer to Figure 7 , this embodiment further provides an example of a vehicle tracking data processing method applying the above method embodiment, including the following steps:

[0137] S1, obtain the target vehicle tracking image, perform whole-image vehicle recognition, and obtain the whole-image vehicle detection frame of the environmental vehicle in the target vehicle tracking image;

[0138] S2, based on the historical vehicle trajectory, obtain the initial perception frame of the environmental vehicle at the previous moment, and obtain the predicted perception frame of the environmental vehicle in the target vehicle tracking image through a preset motion model;

[0139] S3, based on a preset pairing model, perform pairing processing on the initial perception frames corresponding to each environmental vehicle to obtain the target vehicle pairs in the environmental vehicles;

[0140] S4, based on the whole-image vehicle detection frame, the predicted perception frame, and a preset missed detection recognition model, determine whether there is a missed detection of the target vehicle pair in the target vehicle tracking image;

[0141] S5, if there is no missed detection of the target vehicle pair in the target vehicle tracking image, jump to S7;

[0142] S6, there is a missed detection of the target vehicle pair in the target vehicle tracking image;

[0143] S7, perform sub-image vehicle recognition on the target sub-region where the target vehicle pair with a missed detection exists in the target vehicle tracking image to obtain the sub-image vehicle detection frame;

[0144] S8, calculate the cost matrix;

[0145] S9, match the vehicle detection frame with the historical vehicle trajectory through the KM matching algorithm;

[0146] S10, update the preset motion model.

[0147] In practical applications, the preset motion model is usually set based on the KF filter, which includes the historical vehicle trajectory, and updates the historical vehicle trajectory based on the updated vehicle trajectory.

[0148] In this embodiment, the update of the historical vehicle trajectory usually includes the following four types:

[0149] When it is determined that there is no missed detection, based on the calculation results of the KM matching algorithm between the detected vehicle detection boxes of the entire image and the historical vehicle trajectories, each historical vehicle trajectory is updated to obtain the corresponding current vehicle trajectory;

[0150] When it is determined that there is no missed detection, based on the calculation results of the KM matching algorithm between the detected vehicle detection boxes of the entire image and the sub-image vehicle detection boxes and the historical vehicle trajectories, each historical vehicle trajectory is updated to obtain the corresponding current vehicle trajectory;

[0151] Based on the calculation results of the KM matching algorithm, new vehicle tracking trajectories are created for the vehicle detection boxes of the entire image or the sub-image vehicle detection boxes that do not match the historical vehicle trajectories.

[0152] Based on the calculation results of the KM matching algorithm, if there are historical vehicle trajectories that cannot be matched, it is determined whether to delete the unupdated historical vehicle trajectories based on preset conditions.

[0153] It should be noted that the above examples are only examples of applying the above method of this embodiment to perform vehicle tracking data processing, and are used to illustrate the application process of the embodiments of the present application, and cannot constitute a specific limitation on the method embodiments of the present application.

[0154] Based on the above method embodiment, this embodiment further provides a vehicle tracking data processing device. Please refer to Figure 8 , the device includes:

[0155] An image acquisition module 10, configured to acquire a target vehicle tracking image; the target vehicle tracking image is an image acquired by the on-vehicle camera device of the current vehicle at the current moment and including at least two environmental vehicles in front of the vehicle;

[0156] A historical trajectory query module 20, configured to obtain the initial perception frames corresponding to the environmental vehicles in front of the vehicle in the historical vehicle tracking image at the previous moment based on at least two historical vehicle trajectories; the historical vehicle trajectories represent the driving trajectories corresponding to a single environmental vehicle in the historical vehicle tracking image;

[0157] An image target extraction module 30, configured to perform vehicle recognition of the entire image on the target vehicle tracking image based on a preset vehicle perception model to obtain the vehicle detection boxes of the entire image corresponding to the target vehicle tracking image;

[0158] A target vehicle pair detection module 40, configured to perform pairing processing on the initial perception frames corresponding to each environmental vehicle based on a preset pairing model to obtain an environmental vehicle pairing result;

[0159] The trajectory prediction module 50 is configured to, when the environmental vehicle pairing result indicates the existence of a target vehicle pair, predict the positions of the perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle pair in the target vehicle tracking image based on a preset motion model and the initial perception frame corresponding to the target vehicle pair, and obtain the predicted perception frames of the first environmental vehicle and the second environmental vehicle respectively;

[0160] The missed detection judgment module 60 is configured to perform missed detection on the target vehicle pair in the target vehicle tracking image based on the vehicle detection frame of the entire image, the predicted perception frame, and a preset missed detection recognition model;

[0161] The vehicle trajectory update module 70 is configured to, when it is determined that there is no missed detection, update each historical vehicle trajectory based on the vehicle detection frame of the entire image to obtain the corresponding current vehicle trajectory.

[0162] In some embodiments, the image target extraction module 30 further includes a sub-region generation module and a sub-image detection module.

[0163] The sub-region generation module is configured to, when it is determined that there is a missed detection, determine the target sub-region corresponding to the missed target vehicle pair in the target vehicle tracking image.

[0164] The sub-image detection module is configured to perform sub-image target extraction on the target sub-region based on a preset vehicle perception model to obtain the sub-image vehicle detection frame corresponding to the target sub-region.

[0165] The vehicle trajectory update module 70 is further configured to update each historical vehicle trajectory based on the vehicle detection frame of the entire image and the sub-image vehicle detection frame to obtain the corresponding current vehicle trajectory.

[0166] In some embodiments, the sub-region generation module is further configured to obtain the predicted perception frame corresponding to the first environmental vehicle and the predicted perception frame corresponding to the second environmental vehicle in the missed target vehicle pair; perform a magnification process on the predicted perception frame with a larger area among the predicted perception frames of the first environmental vehicle and the second environmental vehicle to obtain the target sub-region.

[0167] In some embodiments, the vehicle trajectory update module 70 is further configured to, when an un-matched vehicle detection frame of the entire image or a sub-image vehicle detection frame is detected in the target vehicle tracking image, create a new vehicle tracking trajectory based on the un-matched vehicle detection frame of the entire image or the sub-image vehicle detection frame.

[0168] In some embodiments, the vehicle trajectory update module 70 is further configured to, when the environmental vehicle pairing result indicates the non-existence of a target vehicle pair, update each historical vehicle trajectory based on the vehicle detection frame of the entire image to obtain the corresponding current vehicle trajectory.

[0169] In some embodiments, when the environmental vehicle pairing result indicates the existence of a target vehicle pair, the target vehicle pair detection module 40 is further configured to respectively obtain the bottom position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the target vehicle pair; determine the larger and smaller values of the bottom position coordinates of the initial perception frames of the first environmental vehicle and the second environmental vehicle in the vehicle body height direction; determine the environmental vehicle corresponding to the smaller value as the leading vehicle and the environmental vehicle corresponding to the larger value as the trailing vehicle; the leading vehicle represents the environmental vehicle closest to the current vehicle in the field of view of the vehicle-mounted camera device, and the trailing vehicle represents the environmental vehicle whose distance from the current vehicle exceeds that of the leading vehicle.

[0170] In some embodiments, the target vehicle pair detection module 40 is further configured to obtain the position coordinates of the initial perception frame in the historical vehicle tracking image at the previous moment in the camera coordinate system; obtain the corresponding position coordinates in the vehicle body coordinate system based on the position coordinates in the camera coordinate system; obtain the lateral spacing between environmental vehicles based on the position coordinates in the vehicle body coordinate system; the lateral spacing is the distance between environmental vehicles along the vehicle axis direction in the vehicle body coordinate system; obtain the environmental vehicle pairing result based on the lateral spacing, the intersection over union (IoU) and the width ratio of the two initial perception frames corresponding to the lateral spacing.

[0171] In some embodiments, the missed detection judgment module 60 is further configured to respectively obtain the intersection over union (IoU) of the corresponding predicted perception frame and the whole-image vehicle detection frame of the first environmental vehicle and the second environmental vehicle in the target vehicle pair, to obtain the first set of IoU corresponding to the first environmental vehicle and the second set of IoU corresponding to the second environmental vehicle; obtain the maximum value in the first set of IoU and the first index information of the whole-image vehicle detection frame corresponding thereto; obtain the maximum value in the second set of IoU and the second index information of the whole-image vehicle detection frame corresponding thereto; if the maximum value in the first set of IoU and the maximum value in the second set of IoU are both greater than the preset threshold and the first index information is not equal to the second index information, it is determined that there is no missed detection of the target vehicle pair; otherwise, there is a missed detection.

[0172] The vehicle tracking data processing method of the present application may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement various aspects of the present application.

[0173] The embodiments of the present application further provide an electronic device, which includes a processor and a memory. At least one instruction or at least one segment of program is stored in the memory, and the at least one instruction or the at least one segment of program is loaded and executed by the processor to implement the control method of the airbag provided in the control method embodiment as described above. Figure 9 It is a hardware structure block diagram of an electronic device provided by the embodiments of the present application. As Figure 9As shown, the electronic device 800 can vary significantly due to different configurations or performances. It may include one or more central processing units (CPUs) 810 (the processor 810 may include, but is not limited to, processing devices such as a microprocessor MCU or a field-programmable gate array FPGA), a memory 830 for storing data, and one or more storage media 820 for storing application programs 823 or data 822 (such as one or more mass storage devices). Among them, the memory 830 and the storage media 820 can be transient storage or persistent storage. The program stored in the storage media 820 may include one or more modules, and each module may include a series of instruction operations on the server. Further, the central processor 810 can be configured to communicate with the storage media 820 and execute a series of instruction operations in the storage media 820 on the electronic device 800. The electronic device 800 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input / output interfaces 840, and / or one or more operating systems 821, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so on.

[0174] The input / output interface 840 can be used to receive or send data via a network. Specific examples of the above network may include the wireless network provided by the communication provider of the electronic device 800. In one example, the input / output interface 840 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 840 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0175] Those of ordinary skill in the art can understand that Figure 9 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the electronic device 800 may also include more or fewer components than Figure 9 shown, or have a different configuration from Figure 9 shown.

[0176] The memory can be used to store software programs and modules. By running the software programs and modules stored in the memory, the processor can execute various functional applications and data processing. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device or other volatile solid-state storage devices. Correspondingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0177] The message processing method provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device.

[0178] An embodiment of the present application also provides a computer-readable storage medium. The storage medium can be disposed in the server to store at least one instruction or at least one segment of a program related to the message processing method in the method embodiment. The at least one instruction or the at least one segment of the program is loaded and executed by the processor to implement the above vehicle tracking data processing method.

[0179] Optionally, in this embodiment, the above storage medium can be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the above storage medium can include, but is not limited to: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks or optical discs.

[0180] As can be seen from the embodiments of the vehicle tracking data processing method, device, and storage medium provided by the present application above, the present application obtains an initial perception box corresponding to an environmental vehicle in a historical vehicle tracking image at the previous moment based on historical vehicle trajectories; performs recognition on a target vehicle tracking image based on a preset vehicle perception model to obtain a whole-image vehicle detection box; performs pairing processing on the initial perception box to obtain an environmental vehicle pairing result; if there is a target vehicle pair, obtains a predicted perception box based on the preset motion model and the initial perception box corresponding to the target vehicle pair; performs omission detection on the target vehicle pair based on the whole-image vehicle detection box, the predicted perception box, and a preset omission detection model; and determines an update strategy for historical vehicle trajectories based on the detection result. The target vehicle pair is determined through historical trajectory data and a preset motion model, and it is determined whether there is an omission of the target vehicle pair. In the case of an omission of the target vehicle pair, a target sub-region is defined based on the predicted perception box, and this region is re-detected to improve the vehicle detection accuracy. At the same time, the vehicle recognition of the whole image and the vehicle recognition of the target sub-region are fused based on the target tracking framework, which can improve the time efficiency of vehicle tracking data processing.

[0181] It should be noted that: the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of the present application have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In certain embodiments, multi-tasking and parallel processing are also possible or may be advantageous.

[0182] Each embodiment in the present application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0183] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk, or an optical disc, etc.

[0184] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0185] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

[0186] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A vehicle tracking data processing method, characterized in that, Including: Obtain a target vehicle tracking image; the target vehicle tracking image is an image obtained by an in-vehicle camera device of the current vehicle at the current moment, including images of at least two environmental vehicles in front of the vehicle; Based on at least two historical vehicle trajectories, obtain the initial perception frames corresponding to the environmental vehicles in front of the vehicle in the historical vehicle tracking image at the previous moment; The historical vehicle trajectory represents the driving trajectory corresponding to a single environmental vehicle in the historical vehicle tracking image; Perform full-image vehicle recognition on the target vehicle tracking image based on a preset vehicle perception model to obtain the full-image vehicle detection frame corresponding to the target vehicle tracking image; Perform pairing processing on the initial perception frames corresponding to each environmental vehicle based on a preset pairing model to obtain an environmental vehicle pairing result; In the case where the environmental vehicle pairing result indicates the existence of a target vehicle pair, based on a preset motion model and the initial perception frames corresponding to the target vehicle pair, predict the positions of the perception frames corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle tracking image, respectively, to obtain the predicted perception frames of the first environmental vehicle and the second environmental vehicle; the target vehicle pair refers to the environmental vehicles corresponding to two initial induction frames on different vehicle trajectories when the lateral distance is less than or equal to a preset threshold, the intersection-over-union ratio of the two initial perception frames is greater than 0.5, and the width ratio is greater than 0.7; Perform omission detection on the target vehicle tracking image for the target vehicle pair based on the full-image vehicle detection frame, the predicted perception frame, and a preset omission detection model; the omission detection is used to determine whether two full-image vehicle detection frames corresponding to the target vehicle pair are recognized; In the case of omission detection, fuse the vehicle recognition of the full image and the vehicle recognition of the target sub-region to obtain the current vehicle trajectory; the target sub-region is the region of the target vehicle pair with omission detection in the target vehicle tracking image.

2. The method according to claim 1, wherein The method further includes: In the case of determining that there is omission detection, determine the target sub-region corresponding to the target vehicle pair with omission detection in the target vehicle tracking image; Perform sub-graph target extraction on the target sub-region based on the preset vehicle perception model to obtain the sub-graph vehicle detection frame corresponding to the target sub-region; Update each historical vehicle trajectory based on the full-image vehicle detection frame and the sub-graph vehicle detection frame to obtain the corresponding current vehicle trajectory.

3. The method according to claim 1, wherein The method further includes: In the case of determining that there is no omission detection, update each historical vehicle trajectory based on the full-image vehicle detection frame to obtain the corresponding current vehicle trajectory.

4. The method according to claim 2, wherein The method further includes: in the case where a full-image vehicle detection frame or a sub-graph vehicle detection frame that does not match the historical vehicle trajectory is detected in the target vehicle tracking image, create a new vehicle tracking trajectory based on the full-image vehicle detection frame or the sub-graph vehicle detection frame that does not match the historical vehicle trajectory.

5. The method according to claim 2, wherein The determination of the target sub-region corresponding to the target vehicle pair with omission detection in the target vehicle tracking image includes: Obtain the predicted perception frame corresponding to the first environmental vehicle and the predicted perception frame corresponding to the second environmental vehicle in the target vehicle pair with omission detection; Enlarge the prediction perception box with a larger area in the prediction perception boxes of the first environmental vehicle and the second environmental vehicle to obtain the target sub-region.

6. The method according to claim 1, characterized in that, The method further includes: When the environmental vehicle pairing result indicates the absence of a target vehicle pair, update each historical vehicle trajectory based on the full-image vehicle detection box to obtain the corresponding current vehicle trajectory.

7. The method according to claim 1, characterized in that When the environmental vehicle pairing result indicates the presence of a target vehicle pair, the method further includes: Obtain the bottom edge position coordinates of the initial perception boxes of the first environmental vehicle and the second environmental vehicle in the target vehicle pair respectively; Determine the larger and smaller values of the bottom edge position coordinates of the initial perception boxes of the first environmental vehicle and the second environmental vehicle in the vehicle body height direction; Determine the environmental vehicle corresponding to the smaller value as the leading vehicle, and the environmental vehicle corresponding to the larger value as the trailing vehicle; The leading vehicle represents the environmental vehicle closest to the current vehicle in the field of view of the on-vehicle camera device, and the trailing vehicle represents the environmental vehicle whose distance from the current vehicle exceeds that of the leading vehicle.

8. The method according to claim 1, characterized in that, The pairing process of the initial perception boxes corresponding to each environmental vehicle based on a preset pairing model to obtain the environmental vehicle pairing result includes: Obtain the position coordinates of the initial perception boxes in the historical vehicle tracking image at the previous moment in the camera coordinate system; Based on the position coordinates in the camera coordinate system, obtain the corresponding position coordinates in the vehicle body coordinate system; Based on the position coordinates in the vehicle body coordinate system, obtain the lateral spacing between every two environmental vehicles; the lateral spacing is the distance between every two environmental vehicles along the vehicle axis direction in the vehicle body coordinate system; Obtain the environmental vehicle pairing result based on the lateral spacing, the intersection-over-union ratio and the width ratio of the two initial perception boxes corresponding to the lateral spacing.

9. The method according to claim 1, wherein The omission detection of the target vehicle tracking image for the target vehicle pair based on the full-image vehicle detection box, the prediction perception box and a preset omission detection recognition model includes: Obtain the intersection-over-union ratios of the corresponding prediction perception boxes of the first environmental vehicle and the second environmental vehicle in the target vehicle pair and the full-image vehicle detection box respectively, to obtain the first set of intersection-over-union ratios corresponding to the first environmental vehicle and the second set of intersection-over-union ratios corresponding to the second environmental vehicle; Obtain the maximum value in the first set of intersection-over-union ratios, and the first index information of the full-image vehicle detection box corresponding thereto; Obtain the maximum value in the second set of intersection-over-union ratios, and the second index information of the full-image vehicle detection box corresponding thereto; If both the maximum value in the first set of intersection-over-union ratios and the maximum value in the second set of intersection-over-union ratios are greater than a preset threshold, and the first index information is not equal to the second index information, it is determined that there is no omission detection for the target vehicle pair; otherwise, there is an omission detection.

10. A vehicle tracking data processing device, characterized in that, Includes: An image acquisition module (10) for acquiring a target vehicle tracking image; the target vehicle tracking image is an image including at least two environmental vehicles in front of the vehicle acquired by the on-vehicle camera device of the current vehicle at the current moment; A historical trajectory query module (20) is configured to obtain an initial perception box corresponding to an environmental vehicle in front of the vehicle in a historical vehicle tracking image at a previous moment based on at least two historical vehicle trajectories; the historical vehicle trajectories represent the driving trajectories corresponding to a single environmental vehicle in the historical vehicle tracking image. An image target extraction module (30) is configured to perform full-image vehicle recognition on the target vehicle tracking image based on a preset vehicle perception model to obtain a full-image vehicle detection box corresponding to the target vehicle tracking image. A target vehicle pair detection module (40) is configured to perform pairing processing on the initial perception boxes corresponding to each environmental vehicle based on a preset pairing model to obtain an environmental vehicle pairing result. A trajectory prediction module (50) is configured to, when the environmental vehicle pairing result indicates the existence of a target vehicle pair, predict the positions of the perception boxes corresponding to the first environmental vehicle and the second environmental vehicle in the target vehicle pair in the target vehicle tracking image based on a preset motion model and the initial perception boxes corresponding to the target vehicle pair, respectively obtaining the predicted perception boxes of the first environmental vehicle and the second environmental vehicle; the target vehicle pair refers to the environmental vehicles corresponding to two initial induction boxes on different vehicle trajectories when the lateral distance between the two initial induction boxes is less than or equal to a preset threshold, the intersection over union of the two initial perception boxes is greater than 0.5, and the width ratio is greater than 0.

7. An undetected detection module (60) is configured to perform undetected detection on the target vehicle tracking image for the target vehicle pair based on the full-image vehicle detection box, the predicted perception box, and a preset undetected recognition model; the undetected detection is used to determine whether two full-image vehicle detection boxes corresponding to the target vehicle pair are recognized. A vehicle trajectory update module (70) is configured to, when undetected detection occurs, fuse the vehicle recognition of the full image and the vehicle recognition of the target sub-region to obtain the current vehicle trajectory; the target sub-region is the region of the undetected target vehicle pair in the target vehicle tracking image.

11. A computer-readable storage medium, characterized in that, At least one instruction or at least one program segment is stored in the storage medium, and the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the vehicle tracking data processing method according to any one of claims 1-9.

Images