Vehicle target tracking method, terminal device and storage medium

Abstract

The present invention relates to a vehicle target tracking method, terminal equipment, and storage medium. The method includes: positioning a vehicle in a current frame and using the positioned vehicle as a vehicle to be tracked; when the trajectory information of the vehicle to be tracked is in a historical trajectory list, predicting the predicted position of each vehicle to be tracked based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list; drawing a fan-shaped area that may exist in the current frame; judging whether the actual position of each vehicle to be tracked intersects with the fan-shaped area; if the fan-shaped area intersects with the actual position of only one vehicle to be tracked, setting the vehicle corresponding to the fan-shaped area and the vehicle to be tracked to be the same vehicle; if the fan-shaped area intersects with the actual positions of two or more vehicles to be tracked, screening the vehicles corresponding to the fan-shaped area from the two or more vehicles to be tracked using a Hungarian algorithm. The present invention can improve the tracking effect while ensuring the tracking speed.

Description

Technical Field

[0001] The present invention relates to the field of target tracking, and in particular to a vehicle target tracking method, terminal equipment and storage medium. Background Art

[0002] Existing vehicle target tracking algorithms have the following defects: (1) strong dependence on the environment; (2) the tracking effect of deep learning algorithms with fast tracking speed is poor; (3) the tracking speed of deep learning algorithms with good tracking effect is slow. Therefore, there is an urgent need for a technical solution that can guarantee both tracking speed and tracking effect. Summary of the Invention

[0003] In order to solve the above problems, the present invention proposes a vehicle target tracking method, a terminal device and a storage medium.

[0004] The specific plan is as follows:

[0005] A vehicle target tracking method comprises the following steps:

[0006] S1: Locate the vehicle in the road image of the current frame and use the located vehicle as the vehicle to be tracked;

[0007] S2: For each vehicle to be tracked, determine whether its trajectory information is in the historical trajectory list. If so, proceed to S3; otherwise, add the trajectory information of the vehicle to be tracked to the historical trajectory list, wait for the next frame of road image to be received, and then return to S1;

[0008] S3: Predicting the predicted position of each vehicle to be tracked based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list;

[0009] S4: Draw a sector area where the current frame may exist based on the actual position of each vehicle to be tracked in the three frames before the current frame and the distance between the predicted position of the current frame and the actual position of the previous frame;

[0010] S5: Determine whether the actual position of each vehicle to be tracked intersects with the sector-shaped area. If the sector-shaped area intersects with the actual position of only one vehicle to be tracked, the vehicle corresponding to the sector-shaped area is set to be the same vehicle as the vehicle to be tracked. If the sector-shaped area intersects with the actual positions of more than two vehicles to be tracked, the Hungarian algorithm is used to filter out the same vehicle as the vehicle corresponding to the sector-shaped area from the more than two vehicles to be tracked. Update the historical trajectory list according to the actual position of each vehicle to be tracked in the current frame.

[0011] Furthermore, step S6 is included: when it is determined to be the same vehicle, similarity is calculated based on color, position, and features respectively, and the three similarity calculation results are weighted and summed to obtain the total similarity, and whether it is the same vehicle is determined based on the relationship between the total similarity and the similarity threshold.

[0012] Furthermore, vehicle positioning is performed through a vehicle positioning model. The network structure of the vehicle positioning model is based on the yolov4 network, with the backbone part replaced by the mobilenetv2 structure. The activation function uses prelu, and DIOU_NMS is used for post-processing.

[0013] Furthermore, the parameters of the trajectory information include: x, y, r, w, vx, vy, vr, vw, where x and y represent the x-axis coordinate and y-axis coordinate of the center point of the vehicle position, respectively, r represents the aspect ratio of the vehicle positioning frame, w represents the width of the vehicle positioning frame, and vx, vy, vr, vw represent the increments of x, y, r, and w, respectively.

[0014] Furthermore, based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list, the calculation formula for predicting the predicted position of each vehicle to be tracked is:

[0015]

[0016] Among them, the subscript n represents the nth historical trajectory information of each vehicle to be tracked in the historical trajectory list, x n and y n They represent the x-axis coordinate and y-axis coordinate of the center point of the vehicle position corresponding to the nth historical trajectory information, r n Indicates the width-to-height ratio of the vehicle positioning frame corresponding to the nth historical trajectory information, w n Indicates the width of the vehicle positioning box corresponding to the nth historical trajectory information, vx n vy n vr n vw n They represent the increments of x, y, r, and w of the nth historical trajectory information relative to the n-1th historical trajectory information; a ni Represents the total weight corresponding to the i-th parameter of the n-th historical trajectory information, i∈[1,8];

[0017] The total weight is equal to the product of the proportional weight and the time weight;

[0018] The proportional weight of the i-th parameter in the n-th historical trajectory information is calculated as follows: the i-th parameter in the n-th historical trajectory information is subtracted from the i-th parameter in the predicted position corresponding to the n-th historical trajectory information, the difference calculation result is input into the Gaussian model, and the output of the Gaussian model is used as the proportional weight;

[0019] The time weights of all parameters in each historical trajectory information are the same. The time weight of the nth historical trajectory information is calculated as follows: the reciprocal of the time difference between the acquisition time corresponding to the nth historical trajectory information and the acquisition time corresponding to the current frame is used as the time weight.

[0020] Furthermore, based on the actual positions of the three frames before the current frame of each vehicle to be tracked and the distance between the predicted position of the current frame and the actual position of the previous frame, a method for drawing a sector-shaped area that may exist in the current frame is as follows: after connecting the center points corresponding to the actual positions of the three frames before the current frame and the vehicle orientation in sequence, the angle formed by the connection is used as the central angle of the sector; the distance between the predicted position of the current frame and the actual position of the previous frame is used as the radius of the sector, the actual position of the previous frame is used as the vertex, the vehicle orientation of the previous frame is used as the center line, and the sector is drawn according to the obtained central angle and radius.

[0021] A vehicle target tracking terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above are implemented.

[0022] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described above in an embodiment of the present invention.

[0023] The present invention adopts the above technical solution, which can improve the tracking effect while ensuring the tracking speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Shown is a flow chart of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0025] To further illustrate various embodiments, the present invention provides accompanying drawings. These drawings form part of the present disclosure and are primarily used to illustrate the embodiments and, in conjunction with the relevant description in the specification, to explain the operating principles of the embodiments. By referring to these drawings, those skilled in the art will be able to understand other possible implementations and the advantages of the present invention.

[0026] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0027] Example 1:

[0028] The embodiment of the present invention provides a vehicle target tracking method, such as Figure 1 As shown, the method includes the following steps:

[0029] S1: Locate the vehicle in the road image of the current frame and use the located vehicle as the vehicle to be tracked.

[0030] In this embodiment, positioning is performed using a pre-trained vehicle positioning model, and multiple vehicles may be located in one image.

[0031] The vehicle positioning model's network architecture is based on the YoloV4 network, with the backbone replaced by the MobileNetV2 architecture. Structural optimizations are performed to reduce network depth and width, enabling it to be adapted for vehicle positioning while reducing parameters and computational complexity. Unnecessary branches are removed, improving speed. Prelu is used as the activation function for greater cross-platform portability. DIOU_NMS is used for post-processing.

[0032] Furthermore, during pre-training of the license plate location model in this embodiment, the vehicle image input is augmented using one of the following random augmentation modes: CutMix, GridMask, Fence Mask, and morphological changes (including cropping, distortion, color jittering, brightness enhancement, etc.). Using this data augmentation mode can improve the accuracy of the trained model.

[0033] S2: For each vehicle to be tracked, determine whether its trajectory information is in the historical trajectory list. If so, proceed to S3; otherwise, add the trajectory information of the vehicle to be tracked to the historical trajectory list, wait for the next frame of road image to be received, and then return to S1.

[0034] The historical trajectory list is used to store the actual location information of the vehicle. In this embodiment, the parameters for setting the trajectory information include: x, y, r, w, vx, vy, vr, vw, where x and y represent the x-axis coordinate and y-axis coordinate of the center point of the vehicle position, respectively, r represents the aspect ratio of the vehicle positioning frame, w represents the width of the vehicle positioning frame, and vx, vy, vr, and vw represent the increments of the four parameters x, y, r, and w of the current frame relative to the previous frame, respectively.

[0035] S3: Predicting the predicted position of each vehicle to be tracked based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list.

[0036] In this embodiment, the calculation formula for predicting the predicted position of each vehicle to be tracked based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list is:

[0037]

[0038] Among them, the subscript n represents the nth historical trajectory information of each vehicle to be tracked in the historical trajectory list, x n and yn They represent the x-axis coordinate and y-axis coordinate of the center point of the vehicle position corresponding to the nth historical trajectory information, r n Indicates the width-to-height ratio of the vehicle positioning frame corresponding to the nth historical trajectory information, w n Indicates the width of the vehicle positioning box corresponding to the nth historical trajectory information, vx n vy n vr n vw n They represent the increments of x, y, r, and w of the nth historical trajectory information relative to the n-1th historical trajectory information; a ni Represents the total weight corresponding to the i-th parameter of the n-th historical trajectory information, i∈[1,8].

[0039] The total weight is equal to the product of the proportional weight and the time weight.

[0040] The proportional weight of the i-th parameter in the n-th historical trajectory information is calculated by performing a difference calculation between the i-th parameter in the n-th historical trajectory information and the i-th parameter in the predicted position corresponding to the n-th historical trajectory information, inputting the difference calculation result into the Gaussian model, and using the output of the Gaussian model as the proportional weight.

[0041] The time weights of all parameters in each historical trajectory information are the same. The time weight of the nth historical trajectory information is calculated as follows: the reciprocal of the time difference between the acquisition time corresponding to the nth historical trajectory information and the acquisition time corresponding to the current frame is used as the time weight.

[0042] S4: Draw a sector area where the current frame may exist based on the actual position of each vehicle to be tracked in the three frames before the current frame and the distance between the predicted position of the current frame and the actual position of the previous frame.

[0043] In this embodiment, the center points corresponding to the actual positions of the first three frames are connected in sequence with the vehicle orientation, and the angle formed by the connection is used as the central angle of the sector; the distance between the predicted position of the current frame and the actual position of the previous frame is used as the radius of the sector, the actual position of the previous frame is used as the vertex, and the vehicle orientation of the previous frame is used as the center line, and the sector is drawn according to the obtained central angle and radius.

[0044] S5: Determine whether the actual position of each vehicle to be tracked intersects with the sector-shaped area. If the sector-shaped area intersects with the actual position of only one vehicle to be tracked, the vehicle corresponding to the sector-shaped area is set to be the same vehicle as the vehicle to be tracked. If the sector-shaped area intersects with the actual positions of more than two vehicles to be tracked, the Hungarian algorithm is used to filter out the same vehicle as the vehicle corresponding to the sector-shaped area from the more than two vehicles to be tracked. Update the historical trajectory list according to the actual position of each vehicle to be tracked in the current frame.

[0045] The vehicles are tracked based on the determination result that they belong to the same vehicle.

[0046] To further improve vehicle tracking accuracy, this embodiment includes step S6 after step S5: When the vehicle is determined to be the same, similarity is calculated based on color, position, and features. The three similarity calculation results are weighted and summed to obtain a total similarity. The relationship between the total similarity and the similarity threshold is used to determine whether the vehicle is the same. In this embodiment, the weights for color, position, and features are set to 3, 6, and 1, respectively.

[0047] The embodiments of the present invention have the following beneficial effects:

[0048] (1) The mobilenet structure is used to optimize the positioning network model. Compared with other similar networks, it is faster while taking into account the accuracy of license plate positioning.

[0049] (2) The vehicle position is estimated based on the vehicle position trajectory, making the tracking more stable.

[0050] (3) Vehicle tracking is performed by using location information, color information, vehicle matching and other weighted integrated information to improve tracking accuracy.

[0051] (4) The integrated Hungarian algorithm and the optimal radiation solution of the vehicle trajectory are used to synchronously determine the relationship between the front and rear frames of the vehicle, further improving the tracking accuracy.

[0052] Example 2:

[0053] The present invention also provides a vehicle target tracking terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiment of the first embodiment of the present invention are implemented.

[0054] Furthermore, as an executable solution, the vehicle target tracking terminal device can be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The vehicle target tracking terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above-mentioned component structure of the vehicle target tracking terminal device is merely an example of a vehicle target tracking terminal device and does not constitute a limitation on the vehicle target tracking terminal device. The vehicle target tracking terminal device may include more or fewer components than those described above, or a combination of certain components, or different components. For example, the vehicle target tracking terminal device may also include input and output devices, network access devices, buses, etc., but the embodiments of the present invention do not limit this.

[0055] Furthermore, as an executable solution, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the vehicle target tracking terminal device, and utilizes various interfaces and lines to connect various parts of the entire vehicle target tracking terminal device.

[0056] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the vehicle target tracking terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created based on the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0057] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.

[0058] If the module / unit integrated in the vehicle target tracking terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM) and software distribution medium, etc.

[0059] Although the present invention has been particularly shown and described in conjunction with preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made to the present invention without departing from the spirit and scope of the invention as defined in the appended claims, and all such changes are within the scope of protection of the present invention.

Claims

1. A vehicle target tracking method, characterized in that: The following steps are involved: S1: Locate the vehicle in the road image of the current frame and use the located vehicle as the vehicle to be tracked; S2: For each vehicle to be tracked, determine whether its trajectory information is in the historical trajectory list. If so, proceed to S3; otherwise, add the trajectory information of the vehicle to be tracked to the historical trajectory list, wait for the next frame of road image to be received, and then return to S1; S3: Predicting the predicted position of each vehicle to be tracked based on the historical trajectory information of each vehicle to be tracked in the historical trajectory list; S4: Based on the actual position of each vehicle to be tracked in the previous three frames close to the current frame and the distance between the predicted position of the current frame and the actual position of the previous frame, a sector-shaped area that may exist in the current frame is drawn. The drawing method is as follows: after connecting the center points corresponding to the actual positions of the previous three frames with the vehicle orientation in sequence, the angle formed by the connection is used as the central angle of the sector; the distance between the predicted position of the current frame and the actual position of the previous frame is used as the radius of the sector, the actual position of the previous frame is used as the vertex, and the vehicle orientation of the previous frame is used as the center line, and the sector is drawn according to the obtained central angle and radius; S5: Determine whether the actual position of each vehicle to be tracked in the current frame intersects with the sector-shaped area. If the sector-shaped area intersects with the actual position of only one vehicle to be tracked, the vehicle corresponding to the sector-shaped area is set to be the same vehicle as the vehicle to be tracked. If the sector-shaped area intersects with the actual positions of more than two vehicles to be tracked, the Hungarian algorithm is used to filter the vehicles corresponding to the sector-shaped area from the more than two vehicles to be tracked to be the same vehicle. Update the historical trajectory list according to the actual position of each vehicle to be tracked in the current frame.

2. The vehicle target tracking method according to claim 1, wherein: The method further includes step S6: when it is determined to be the same vehicle, similarity calculations are performed based on color, position, and features respectively, and the three similarity calculation results are weighted and summed to obtain a total similarity, and whether it is the same vehicle is determined based on the relationship between the total similarity and the similarity threshold.

3. The vehicle target tracking method according to claim 1, wherein: Vehicle positioning is performed through the vehicle positioning model. The network structure of the vehicle positioning model is based on the yolov4 network, with the backbone part replaced by the mobilenetv2 structure. The activation function uses prelu, and DIOU_NMS is used for post-processing.

4. The vehicle target tracking method according to claim 1, wherein: The parameters of the trajectory information include: x, y, r, w, vx, vy, vr, vw, where x and y represent the x-axis and y-axis coordinates of the center point of the vehicle position, respectively, r represents the aspect ratio of the vehicle positioning frame, w represents the width of the vehicle positioning frame, and vx, vy, vr, and vw represent the increments of x, y, r, and w, respectively.

5. The vehicle target tracking method according to claim 4, characterized in that: According to the historical trajectory information of each vehicle to be tracked in the historical trajectory list, the calculation formula for predicting the predicted position of each vehicle to be tracked is: Among them, the subscript n represents the nth historical trajectory information of each vehicle to be tracked in the historical trajectory list, x n and y n They represent the x-axis coordinate and y-axis coordinate of the center point of the vehicle position corresponding to the nth historical trajectory information, r n Indicates the width-to-height ratio of the vehicle positioning frame corresponding to the nth historical trajectory information, w n Indicates the width of the vehicle positioning box corresponding to the nth historical trajectory information, vx n vy n vr n vw n They represent the increments of x, y, r, and w of the nth historical trajectory information relative to the n-1th historical trajectory information; a ni Represents the total weight corresponding to the i-th parameter of the n-th historical trajectory information, i∈[1,8]; The total weight is equal to the product of the proportional weight and the time weight; The proportional weight of the i-th parameter in the n-th historical trajectory information is calculated as follows: the i-th parameter in the n-th historical trajectory information is subtracted from the i-th parameter in the predicted position corresponding to the n-th historical trajectory information, the difference calculation result is input into the Gaussian model, and the output of the Gaussian model is used as the proportional weight; The time weights of all parameters in each historical trajectory information are the same. The time weight of the nth historical trajectory information is calculated as follows: the reciprocal of the time difference between the acquisition time corresponding to the nth historical trajectory information and the acquisition time corresponding to the current frame is used as the time weight.

6. A vehicle target tracking terminal device, characterized by: The method comprises a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images