Vehicle tracking method, device, terminal and computer-readable storage medium

Abstract

The present invention provides a vehicle tracking method, device, terminal and computer-readable storage medium. The vehicle tracking method includes performing vehicle part detection on a current frame image in a video to be detected, determining vehicle part detection information of at least one target vehicle contained in the current frame image; predicting vehicle part prediction information of each target vehicle in the current frame image based on the position information of each target vehicle in the historical frame image; determining vehicle part detection information corresponding to the target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image; and determining the position information of the target vehicle in the current frame image based on the vehicle part detection information corresponding to the target vehicle. The present application determines the position information of the target vehicle in the current frame image based on the detected vehicle part detection information, so as to avoid the target vehicle being partially blocked and the position information not being detected, thereby affecting the stability and coherence of the target vehicle tracking.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle tracking, and in particular to a vehicle tracking method, device, terminal and computer-readable storage medium. Background Art

[0002] Object detection and object tracking are key directions of visual tasks. The main tasks of object detection include locating and classifying target vehicles; object tracking is to continuously assign the same identity number (IDentity, ID) to the same target vehicle. Object detection and object tracking, as cutting-edge technologies of artificial intelligence, have been widely used in many fields such as intelligent assisted driving, intelligent security, and intelligent management.

[0003] In the intelligent management of vehicles, due to the installation position restrictions of image acquisition equipment, vehicles parked on the roadside in the captured videos or images often overlap densely. In real life, there are many different types of vehicles. If a taller vehicle is parked closer to the camera, it will cause serious occlusion of the vehicles behind, and sometimes even complete occlusion. In such a scene, dense occlusion and dense overlapping occlusion between vehicles will cause unstable and discontinuous output of the vehicle detection frame, which will affect vehicle tracking. Summary of the invention

[0004] The main technical problem solved by the present invention is to provide a vehicle tracking method, device, terminal and computer-readable storage medium to solve the problems of instability and incoherence of vehicle tracking trajectories in the prior art.

[0005] To solve the above technical problems, the first technical solution adopted by the present invention is: to provide a vehicle tracking method, the vehicle tracking method comprising: performing vehicle part detection on a current frame image in a video to be detected, and determining vehicle part detection information of at least one target vehicle contained in the current frame image; predicting vehicle part prediction information of each target vehicle in the current frame image according to the position information of each target vehicle in a historical frame image, the historical frame image including a video frame before the current frame image in the video to be detected; determining vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image; and determining the position information of each target vehicle in the current frame image based on the vehicle part detection information corresponding to each target vehicle.

[0006] In order to solve the above technical problems, the second technical solution adopted by the present invention is: to provide a vehicle tracking device, the vehicle tracking device includes: a detection module, which is used to detect the vehicle part of the current frame image in the video to be detected, and determine the vehicle part detection information of at least one target vehicle contained in the current frame image; a prediction module, which is used to predict the vehicle part prediction information of each target vehicle in the current frame image according to the position information of each target vehicle in the historical frame image of at least one target vehicle, and the historical frame image includes the video frame before the current frame image in the video to be detected; a matching module, which is used to determine the vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image; and an output module, which is used to determine the position information of each target vehicle in the current frame image based on the vehicle part detection information corresponding to each target vehicle.

[0007] To solve the above technical problems, the third technical solution adopted by the present invention is: providing a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is used to execute program data to implement the steps in the above vehicle tracking method.

[0008] To solve the above technical problems, the fourth technical solution adopted by the present invention is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above vehicle tracking method are implemented.

[0009] The beneficial effects of the present invention are as follows: different from the prior art, a vehicle tracking method, device, terminal and computer-readable storage medium are provided, wherein the vehicle tracking method comprises performing vehicle part detection on a current frame image in a video to be detected, and determining vehicle part detection information of at least one target vehicle contained in the current frame image; predicting vehicle part prediction information of each target vehicle in the current frame image according to position information of each target vehicle in a historical frame image of at least one target vehicle, wherein the historical frame image comprises a video frame before the current frame image in the video to be detected; determining vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image; and determining position information of each target vehicle in the current frame image based on the vehicle part detection information corresponding to each target vehicle. The present application performs vehicle part detection on the current frame image in the video to be detected to obtain unobstructed vehicle part detection information, determines the corresponding vehicle part detection information in the current frame image based on the position information of each target vehicle in the historical frame image before the current frame image, and then determines the position information of the target vehicle in the current frame image according to the determined vehicle part detection information, so as to avoid the target vehicle being partially blocked and the position information not being detected, which affects the stability and continuity of target vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 It is a flow chart of the vehicle tracking method provided by the present invention;

[0012] Figure 2 It is a flow chart of an embodiment of a vehicle tracking method provided by the present invention;

[0013] Figure 3 It is a flow chart of a specific embodiment of the vehicle tracking method provided by the present invention;

[0014] Figure 4 is the detection result of the current frame image in the first embodiment;

[0015] Figure 5 is the detection result of the current frame image in the second embodiment;

[0016] Figure 6 yes Figure 2 A flow chart of a specific embodiment of step S24 in the vehicle tracking method provided;

[0017] Figure 7 yes Figure 6 A flowchart of a specific embodiment of step S241 in the vehicle tracking method provided;

[0018] Figure 8 is the detection result of the current frame image in the third embodiment;

[0019] Fig. 9 yes Figure 6 A flowchart of a specific embodiment of step S242 in the vehicle tracking method provided;

[0020] Fig.10 yes Figure 6 A flowchart of a specific embodiment of step S243 in the vehicle tracking method provided;

[0021] Fig.11 yes Figure 6 A flowchart of a specific embodiment of step S244 in the vehicle tracking method provided;

[0022] Fig.12 yes Figure 6A flowchart of a specific embodiment of step S245 in the vehicle tracking method provided;

[0023] Fig.13 yes Figure 6 A flowchart of a specific embodiment of step S246 in the vehicle tracking method provided;

[0024] Fig.14 yes Figure 6 A flowchart of a specific embodiment of step S247 in the vehicle tracking method provided;

[0025] Fig.15 yes Figure 6 A flowchart of a specific embodiment of step S248 in the vehicle tracking method provided;

[0026] Fig.16 Step S24 is Figure 5 The result of detecting the current frame image in ;

[0027] Fig.17 is the detection result of the current frame image in the fourth embodiment;

[0028] Fig.18 yes Figure 2 A schematic flow chart of a specific embodiment of step S27 in the vehicle tracking method provided;

[0029] Fig.19 is the detection result of the current frame image in the fifth embodiment;

[0030] Fig. 20 yes Figure 2 A flowchart of a specific embodiment of step S28 in the vehicle tracking method provided;

[0031] Fig.21 is a schematic block diagram of an embodiment of a vehicle tracking device provided by the present invention;

[0032] Fig. 22 is a schematic block diagram of an implementation manner of a terminal provided by the present invention;

[0033] Fig.23 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0034] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0035] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0036] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention, a vehicle tracking method provided by the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0038] In scenes with dense overlapping occlusions, the YOLO target detection algorithm is used to perform target detection on the acquired images. In the post-processing process of the detection, non-maximum suppression (NMS) is used to remove redundant vehicle detection frames, which results in the real and effective vehicle detection frames being filtered out. Some vehicles do not have corresponding vehicle detection frames in the image, which in turn affects the vehicle tracking effect and has a very large impact on the accuracy of the entire application.

[0039] The existing roadside vehicle multi-target tracking algorithms can be roughly divided into two types: one is the metric learning detection and tracking method based on automatic feature extraction, such as DeepSORT (Simple Online And Realtime Tracking), FairMOT and other multi-target detection algorithms, but metric learning requires a large number of images of different perspectives for individual targets to be annotated, which is difficult to apply to cost-constrained occasions in the transportation field. The second is the target-based tracking method, such as deep learning target detection models, such as IOUTrack and SORT, etc. These algorithms directly use the target detection model with strong generalization performance as the basis for target tracking. On this basis, the target is tracked by using the target position matching method, which has the core advantages of good real-time performance and low application cost. At present, this type of algorithm is most widely used in intelligent vehicle-road systems. However, in scenes with dense overlapping occlusion, the vehicle contours are highly similar, and the extracted features are easily mixed and difficult to distinguish; therefore, the use of this solution in actual engineering applications may be limited, and it cannot solve the problems of instability and incoherence of vehicle tracking trajectories.

[0040] See also Figure 1 , Figure 1 The figure is a flow chart of the vehicle tracking method provided by the present invention. In this embodiment, a vehicle tracking method is provided, and the vehicle tracking method comprises the following steps.

[0041] S11: Perform vehicle part detection on the current frame image in the video to be detected, and determine vehicle part detection information of at least one target vehicle contained in the current frame image.

[0042] Specifically, a current frame image in a video stream is obtained. The road area or parking area is monitored by image acquisition devices set on both sides of the road or in the parking lot, so that the image acquisition device can obtain a video stream of a dense scene. The video stream contains multiple continuous video frames, and one frame of image is selected as the current frame image. The current frame image contains at least one target vehicle. Among them, the image acquisition device is set at a higher position, and the setting height of the image acquisition device exceeds the height of the vehicle. In this embodiment, the current frame image contains multiple target vehicles, and there is a phenomenon of mutual occlusion between the multiple target vehicles.

[0043] Specifically, the current frame image in the video to be detected is subjected to vehicle multi-part detection to obtain at least one component category image set corresponding to at least one target vehicle contained in the current frame image, wherein the component category image set includes multiple component images of the same category; based on the at least one component category image set, the roof information of each target vehicle is determined. The roof information includes the roof detection frame and the position information of the roof detection frame.

[0044] In one embodiment, component images in the component category image set are screened based on similarities between component images in the same component category image set and confidences of the component images; and roof information of each target vehicle is determined based on at least one screened component category image set.

[0045] In one embodiment, the component category image set includes at least one of a roof image set, a body image set, a front image set, a rear image set, and a license plate image set; the component images in the roof image set, the body image set, the front image set, the rear image set, and the license plate image set are respectively associated to determine that the mutually associated component images correspond to the same target vehicle; based on the component image corresponding to the target vehicle, the roof information of the target vehicle is determined.

[0046] In a preferred embodiment, the license plate image set includes at least one license plate detection frame, and the vehicle rear image set includes at least one vehicle rear detection frame; the overlap rate between each license plate detection frame in the license plate image set and each vehicle rear detection frame in the vehicle rear image set is calculated; in response to the overlap rate between the license plate detection frame and the vehicle rear detection frame exceeding a first preset overlap rate, the license plate detection frame and the vehicle rear detection frame corresponding to the overlap rate are associated.

[0047] In one embodiment, in response to the overlap rates corresponding to a license plate detection frame and multiple rear vehicle detection frames respectively exceeding a first preset overlap rate, the lowest edge lines of multiple rear vehicle detection frames associated with the license plate detection frame are compared; and the association relationship between the rear vehicle detection frame and the license plate detection frame to which the lowest edge line that is not closest to the lower edge line of the current frame image belongs is released.

[0048] In one embodiment, the vehicle front image set includes at least one vehicle front detection frame; in response to the overlap rate between the license plate detection frame and the vehicle rear detection frame being lower than the first preset overlap rate, the overlap rate between the unassociated license plate detection frame and each vehicle front detection frame in the vehicle front image set is calculated; in response to the overlap rate between the license plate detection frame and the vehicle front detection frame exceeding the second preset overlap rate, the license plate detection frame and the vehicle front detection frame corresponding to the overlap rate are associated. In response to the overlap rates corresponding to a license plate detection frame and multiple vehicle front detection frames respectively exceeding the second preset overlap rate, the lowest edges of the multiple vehicle front detection frames associated with the license plate detection frame are compared; and the association relationship between the vehicle front detection frame and the license plate detection frame to which the lowest edge that is not closest to the lower edge of the current frame image belongs is released.

[0049] In one embodiment, the vehicle body image set includes at least one vehicle body detection frame, and the overlap rate between the vehicle rear detection frame and each vehicle body detection frame in the vehicle body image set is calculated; in response to the overlap rate between the vehicle rear detection frame and the vehicle body detection frame exceeding a third preset overlap rate, the vehicle rear detection frame corresponding to the overlap rate is associated with the vehicle body detection frame.

[0050] In one embodiment, in response to the overlap rate between the rear vehicle detection frame and the vehicle body detection frame not exceeding the third preset overlap rate, the overlap rate between the unassociated vehicle body detection frame and each vehicle front detection frame is calculated; in response to the overlap rate between the unassociated vehicle body detection frame and the vehicle front detection frame exceeding the fourth preset overlap rate, the vehicle body detection frame and the vehicle front detection frame corresponding to the overlap rate are associated.

[0051] In one embodiment, in response to the overlap rate between the unassociated vehicle body detection frame and the vehicle front detection frame not exceeding the fourth preset overlap rate, the overlap rate between the unassociated vehicle body detection frame and each license plate detection frame is calculated; in response to the overlap rate between the unassociated vehicle body detection frame and the license plate detection frame exceeding the fifth preset overlap rate, the unassociated vehicle body detection frame is associated with the license plate detection frame.

[0052] In one embodiment, the roof image set includes at least one roof detection frame, and the overlap ratios of the body detection frame and each roof detection frame in the roof image set are calculated; in response to the overlap ratio between the body detection frame and the roof detection frame exceeding a sixth preset overlap ratio, the body detection frame is associated with the roof detection frame. Preferably, the overlap ratio between the part of the body detection frame near the roof and the roof detection frame is calculated; in response to the overlap ratio between the part of the body detection frame near the roof and the roof detection frame exceeding a sixth preset overlap ratio, the body detection frame is associated with the roof detection frame.

[0053] In one embodiment, in response to the target vehicle not detecting a corresponding body detection frame, a body detection frame of the target vehicle is generated based on at least one of the roof detection frame, the front detection frame, the rear detection frame and the license plate detection frame according to the positional relationship between the roof, the front, the rear and the license plate and the body, respectively.

[0054] Based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image, it is determined whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct.

[0055] In one embodiment, the body color similarity, body texture similarity and body width and height similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle are calculated; based on at least one of the body color similarity, body texture similarity and body width and height similarity, the similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle is determined; in response to the similarity evaluation value exceeding the similarity evaluation value threshold, it is determined that the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is correct.

[0056] In one embodiment, in response to the similarity evaluation value not exceeding the similarity evaluation value threshold, the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is released. Based on the weighted sum of the body color similarity, the body texture similarity, and the body width and height similarity, the similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle is determined.

[0057] In one embodiment, in response to the overlap rate between the vehicle body detection frame and the roof detection frame not exceeding the sixth preset overlap rate, the overlap rate between the unassociated roof detection frame and the rear vehicle detection frame is calculated; in response to the overlap rate between the unassociated roof detection frame and the rear vehicle detection frame exceeding the seventh preset overlap rate, the unassociated roof detection frame and the rear vehicle detection frame corresponding to the overlap rate are associated.

[0058] In one embodiment, in response to the overlap rate between the unassociated roof detection frame and the rear vehicle detection frame not exceeding the seventh preset overlap rate, the overlap rate between the unassociated roof detection frame and the front vehicle detection frame is calculated; in response to the overlap rate between the unassociated roof detection frame and the front vehicle detection frame exceeding the eighth preset overlap rate, the unassociated roof detection frame and the front vehicle detection frame corresponding to the overlap rate are associated.

[0059] In one embodiment, in response to the target vehicle not detecting a corresponding roof detection frame, a roof detection frame of the target vehicle in the current frame image is generated based on at least one corresponding position information of the body detection frame, the front detection frame, the rear detection frame and the license plate detection frame according to the positional relationship between the body, the front, the rear and the license plate and the roof, respectively.

[0060] S12: predicting vehicle position prediction information of each target vehicle in a current frame image based on position information of each target vehicle in at least one target vehicle in a historical frame image, where the historical frame image includes a video frame before the current frame image in the video to be detected.

[0061] Specifically, according to the position information of the roof detection frame of each target vehicle in the historical frame image before the current frame image, the roof prediction frame of the target vehicle in the current frame image is predicted.

[0062] S13: Based on the vehicle part prediction information of each target vehicle in the current frame image, determine the vehicle part detection information corresponding to each target vehicle in the current frame image.

[0063] Specifically, according to the matching degree between the roof prediction frame of the target vehicle in the current frame image and each roof detection frame, it is determined whether the target vehicle corresponding to the roof prediction frame is the same as the target vehicle corresponding to the roof detection frame.

[0064] In one embodiment, a center loss value between a center point position of a roof prediction frame and a center point position of a roof detection frame is calculated; a width and height loss value between width and height information of the roof prediction frame and width and height information of the roof detection frame is calculated; based on at least one of the center loss value and the width and height loss value, a loss value between the roof detection frame and the roof prediction frame is determined; in response to the loss value between the roof detection frame and the roof prediction frame being less than a loss value threshold, it is determined that the roof detection frame corresponding to the loss value and the roof prediction frame correspond to the same target vehicle.

[0065] In a specific embodiment, the loss value between the roof detection box and the roof prediction box is determined based on the weighted sum of the center loss value and the width and height loss values.

[0066] S14: Based on the vehicle part detection information corresponding to each target vehicle, determine the position information of each target vehicle in the current frame image.

[0067] Specifically, the position information of the target vehicle in the current frame image is determined according to the vehicle body detection frame associated with the roof detection frame corresponding to the target vehicle.

[0068] In a specific embodiment, the position information of the target vehicle in the current frame image is updated to the trajectory information set of the target vehicle.

[0069] The vehicle tracking method provided in this embodiment includes acquiring a current frame image; the current frame image contains at least one target vehicle; performing vehicle part detection on the current frame image to determine vehicle part detection information of at least one target vehicle; predicting vehicle part prediction information of the target vehicle in the current frame image based on the position information of the target vehicle in the historical frame image before the current frame image; determining vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of the target vehicle in the current frame image; and determining the position information of the target vehicle in the current frame image based on the vehicle part detection information corresponding to the target vehicle. The present application performs vehicle part detection on the acquired current frame image to obtain unobstructed vehicle part detection information, determines the corresponding vehicle part detection information in the current frame image based on the position information of each target vehicle in the historical frame image before the current frame image, and then determines the position information of the target vehicle in the current frame image based on the determined vehicle part detection information, thereby avoiding the stability and continuity of target vehicle tracking being affected by the failure to detect position information due to partial obstruction of the target vehicle.

[0070] See also Figure 2 and Figure 3 , Figure 2 It is a flow chart of an embodiment of a vehicle tracking method provided by the present invention; Figure 3 The figure is a flow chart of a specific embodiment of the vehicle tracking method provided by the present invention. In this embodiment, a vehicle tracking method is provided, and the vehicle tracking method comprises the following steps.

[0071] S21: Get the current frame image in the video stream.

[0072] Specifically, a current frame image in a video stream is obtained. The road area or parking area is monitored by image acquisition devices set on both sides of the road or in the parking lot, so that the image acquisition device can obtain a video stream of a dense scene. The video stream contains multiple continuous video frames, and one frame of image is selected as the current frame image. The current frame image contains at least one target vehicle. Among them, the image acquisition device is set at a higher position, and the setting height of the image acquisition device exceeds the height of the vehicle. In this embodiment, the current frame image contains multiple target vehicles, and there is a phenomenon of mutual occlusion between the multiple target vehicles.

[0073] S22: Perform vehicle multi-part detection on the current frame image to obtain at least one component category image set corresponding to at least one target vehicle.

[0074] Specifically, the current frame image is input into the target detection model to perform vehicle multi-component detection, and the component detection frames of all target vehicles in the current frame image are obtained, and the specific component detection frames include component images. The component images may include at least one of a roof image, a body image, a front image, a rear image, and a license plate image. In other embodiments, the component images may also include other component images, such as a door image, a door handle image, etc. The component detection frame is a rectangular frame.

[0075] The component detection frames corresponding to the component images of the same category are assigned to the same image set to obtain a component category image set. That is, the component category image set contains multiple component images of the same category. Specifically, all the body detection frames detected in the current frame image are assigned to the same image set to obtain a body category image set; all the roof detection frames detected in the current frame image are assigned to the same image set to obtain a roof category image set; all the front detection frames detected in the current frame image are assigned to the same image set to obtain a front category image set; all the rear detection frames detected in the current frame image are assigned to the same image set to obtain a rear category image set; all the license plate detection frames detected in the current frame image are assigned to the same image set to obtain a license plate category image set. The component category image set includes at least one of the roof image set, the body image set, the front image set, the rear image set, and the license plate image set.

[0076] The target detection model in this embodiment is specifically a multi-component detection model, which is trained using multiple images containing the vehicle body, multiple images containing the roof, multiple images containing the front of the vehicle, multiple images containing the rear of the vehicle, and multiple images containing the license plate. The trained multi-component detection model can simultaneously detect and obtain component images of multiple categories of the target vehicle. Specifically, the multi-component detection model is trained based on the component images of the categories to be detected to obtain the required multi-component detection model. Among them, the multi-component detection model can be a YOLO-v3 network model, and other network models can also be selected according to actual conditions, and there is no specific limitation again.

[0077] See also Figure 4 and Figure 5 , Figure 4 is the detection result of the current frame image in the first embodiment; Figure 5 It is the detection result of the current frame image in the second embodiment.

[0078] like Figure 4 As shown in , multiple components are detected for the current frame image, and the body detection frame and license plate detection frame of each target vehicle are detected at the same time. Figure 5As shown, a multi-component detection is performed on the acquired current frame image to obtain a roof detection frame, a body detection frame, a front detection frame, a rear detection frame and / or a license plate detection frame corresponding to each target vehicle.

[0079] S23: Filtering the component images in the component category image set based on the similarity between the component images in the same component category image set and the confidence of the component images.

[0080] Specifically, since the same component of the same target vehicle may correspond to multiple component detection frames, in order to reduce the workload, the detected component detection frames are screened for similarity and component images with higher confidence are retained.

[0081] The component detection frame detected in the above step S22 includes the confidence level and the position coordinates of the component detection frame in the current frame image.

[0082] In a specific embodiment, in a component category image set, the overlap rate between the component detection frames is calculated based on the position coordinates of each component detection frame, and then the similarity between the component images contained in the component detection frames is determined. In response to the similarity between the component detection frames exceeding the preset similarity, it is determined that the two component detection frames corresponding to the similarity correspond to the same component of the same target vehicle, and the one with a larger confidence of the two component detection frames is retained, and the one with a smaller confidence is deleted.

[0083] All component category image sets and all component detection frames in the component category image sets are traversed by the above method.

[0084] All the component detection frames in the component category image set obtained through the above steps belong to different target vehicles.

[0085] In order to further improve the detection accuracy, the falsely detected component detection frames are screened, and the confidence of the component detection frames retained in the component category image set is compared with the preset confidence. In response to the confidence exceeding the preset confidence, the component detection frame corresponding to the confidence is retained; in response to the confidence not exceeding the preset confidence, the component detection frame corresponding to the confidence is deleted.

[0086] All component category image sets and all component detection frames in the component category image sets are traversed by the above method.

[0087] S24: Correlating component images in the roof image set, the vehicle body image set, the front image set, the rear image set, and the license plate image set respectively, and determining that the correlated component images correspond to the same target vehicle.

[0088] Specifically, the license plate image set includes at least one license plate detection frame, the rear image set includes at least one rear detection frame; the front image set includes at least one front detection frame; the body image set includes at least one body detection frame; and the roof image set includes at least one roof detection frame.

[0089] Since the target vehicle in the current frame image is in a densely overlapping scene, the target vehicle may be blocked. If the target vehicle is tracked based on the same vehicle component. When the vehicle component is blocked in the current frame image, the position information of the target vehicle cannot be output in the current frame image, and the tracking trajectory of the target vehicle will be incoherent. Different components corresponding to the same vehicle have a fixed relative position relationship. The component image used to determine the position information of the target vehicle is predicted based on the component image obtained by vehicle detection.

[0090] It is necessary to associate all detected component images belonging to the same target vehicle in the current frame image to form an overall output, so as to predict and determine the component image of the target vehicle's position information. It is also possible to track the target vehicle based on the position information of multiple parts, thereby improving the tracking stability and accuracy of the target vehicle.

[0091] The specific steps of associating all detected component images of the same target vehicle are as follows.

[0092] like Figure 6 As shown, Figure 6 yes Figure 2 A flowchart of a specific embodiment of step S24 in the vehicle tracking method is provided.

[0093] S241: Associating each license plate detection frame in the license plate image set with each vehicle rear detection frame in the vehicle rear image set.

[0094] Specifically, the specific steps of associating each license plate detection frame in the license plate image set with each vehicle rear detection frame in the vehicle rear image set are as follows.

[0095] like Figure 7 As shown, Figure 7 yes Figure 6 A flowchart of a specific embodiment of step S241 in the vehicle tracking method is provided.

[0096] S2411: Calculate the overlap rate between each license plate detection frame in the license plate image set and each vehicle rear detection frame in the vehicle rear image set.

[0097] Specifically, in order to ensure the incomplete conditions of vehicle parts in some special scenes, the license plate detection frame is first associated with the rear vehicle detection frame. The intersection over union (IOU) between each license plate detection frame in the license plate image set and each rear vehicle detection frame in the rear vehicle image set is calculated, and then the overlap rate between the license plate detection frame and the rear vehicle detection frame is determined. In order to determine whether the license plate detection frame and the rear vehicle detection frame correspond to the same target vehicle, the calculated intersection over union is compared with the first preset overlap rate. Among them, the first preset overlap rate is the intersection over union ratio between the license plate detection frame and the rear vehicle detection frame of the same vehicle.

[0098] S2412: In response to the overlap ratio between the license plate detection frame and the vehicle rear detection frame exceeding a first preset overlap ratio, the license plate detection frame and the vehicle rear detection frame corresponding to the overlap ratio are associated.

[0099] Specifically, in response to the intersection-and-union ratio between the license plate detection frame and the rear vehicle detection frame exceeding a first preset overlap rate, it is determined that the license plate detection frame and the rear vehicle detection frame corresponding to the intersection-and-union ratio belong to the same target vehicle, and the license plate detection frame and the rear vehicle detection frame corresponding to the intersection-and-union ratio are associated.

[0100] Traverse all the license plate detection frames in the license plate image set and all the rear vehicle detection frames in the rear vehicle image set, and associate the license plate detection frame and the rear vehicle detection frame corresponding to the same target vehicle.

[0101] S2413: In response to the overlap rates corresponding to a license plate detection frame and multiple vehicle rear detection frames respectively exceeding a first preset overlap rate, the lowest edge lines of multiple vehicle rear detection frames associated with the license plate detection frame are compared.

[0102] Specifically, since the license plate detection frame is smaller than the rear vehicle detection frame, if multiple target vehicles are densely occluded, one license plate detection frame will be associated with multiple rear vehicle detection frames at the same time. If one license plate detection frame is associated with multiple rear vehicle detection frames at the same time, the association relationship between each pair of license plate detection frames and rear vehicle detection frames may be incorrect, and it is necessary to determine the rear vehicle detection frame that has the correct association relationship with the license plate detection frame.

[0103] All the rear detection frames associated with the same license plate detection frame are obtained, and the obtained license plate detection frames are input into the intra-class occlusion relationship model. The intra-class occlusion relationship model compares the lowest edge lines of all the rear detection frames associated with the same license plate detection frame. Since the rear detection frame is a rectangular frame, the lowest edge line of the rear detection frame is the edge line of the rear detection frame closest to the lower edge line of the current frame image.

[0104] like Figure 8 As shown, Figure 8is the detection result of the current frame image in the third embodiment. The rear detection frames and license plate detection frames of different target vehicles in the current frame image are detected. However, although two different target vehicles correspond to different license plate detection frames, the license plates corresponding to the license plate detection frames are the same. It can be seen that two different target vehicles correspond to the same license plate detection frame. By comparing the distance between the lowest edge line of different rear detection frames corresponding to the same license plate detection frame and the lowest edge of the current frame image, the rear detection frame corresponding to the same target vehicle as the license plate detection frame is determined.

[0105] S2414: Disconnecting the association between the vehicle rear detection frame and the license plate detection frame to which the lowest edge line that is not closest to the lower edge line of the current frame image belongs.

[0106] Specifically, the rear detection frame to which the lowest edge of the rear detection frame is closest to the lower edge of the current frame image belongs is selected for matching with the license plate detection frame. It is determined that the rear detection frame to which the lowest edge is closest to the lower edge of the current frame image belongs and the license plate detection frame correspond to the same target vehicle. It is determined that the rear detection frame to which the lowest edge that is not closest to the lower edge of the current frame image belongs and the license plate detection frame correspond to different target vehicles, and the association relationship between the rear detection frame and the license plate detection frame corresponding to different target vehicles is released.

[0107] The associated license plate detection frame and vehicle rear detection frame are marked with the same ID.

[0108] S242: Associating each unassociated license plate detection frame in the license plate image set with each vehicle head detection frame in the vehicle head image set.

[0109] Since it is impossible to detect the same license plate in the same frame with both the rear and the front of the vehicle in one image frame, once the license plate detection frame is determined to be associated with the rear detection frame, there is no need to associate the license plate detection frame that is associated with the rear detection frame, thereby reducing the workload.

[0110] Specifically, the specific steps of associating the unassociated license plate detection frames in the license plate image set with each vehicle head detection frame in the vehicle head image set are as follows.

[0111] like Fig. 9 As shown, Fig. 9 yes Figure 6 A flowchart of a specific embodiment of step S242 in the vehicle tracking method is provided.

[0112] S2421: In response to the overlap ratio between the license plate detection frame and the vehicle rear detection frame being lower than a first preset overlap ratio, the overlap ratio between the unassociated license plate detection frame and each vehicle front detection frame in the vehicle front image set is calculated.

[0113] Specifically, the IOUs corresponding to each license plate detection frame that is not associated with the rear detection frame in the license plate image set and each front detection frame in the front image set are calculated, and then the overlap rate between each license plate detection frame and the front detection frame is determined. In order to determine whether the license plate detection frame and the front detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the second preset overlap rate. The second preset overlap rate is the intersection-and-union ratio between the license plate detection frame and the front detection frame of the same vehicle.

[0114] S2422: In response to the overlap rate between the license plate detection frame and the vehicle front detection frame exceeding a second preset overlap rate, the license plate detection frame and the vehicle front detection frame corresponding to the overlap rate are associated.

[0115] Specifically, in response to the intersection-and-union ratio between the license plate detection frame and the vehicle front detection frame exceeding a second preset overlap rate, it is determined that the license plate detection frame and the vehicle front detection frame corresponding to the intersection-and-union ratio belong to the same target vehicle, and the license plate detection frame corresponding to the intersection-and-union ratio is associated with the vehicle front detection frame.

[0116] Traverse all license plate detection frames in the license plate image set that are not associated with the vehicle rear detection frame and all vehicle front detection frames in the vehicle front image set, and associate the license plate detection frame corresponding to the same target vehicle with the vehicle front detection frame.

[0117] S2423: In response to the overlap rates corresponding to a license plate detection frame and multiple vehicle front detection frames respectively exceeding a second preset overlap rate, the lowest edge lines of the multiple vehicle front detection frames associated with the license plate detection frame are compared.

[0118] Specifically, since the license plate detection frame is smaller than the vehicle head detection frame, if multiple target vehicles are densely occluded, a license plate detection frame will be associated with multiple vehicle head detection frames at the same time. If a license plate detection frame is associated with multiple vehicle head detection frames at the same time, the association relationship between each pair of license plate detection frames and vehicle head detection frames may be incorrect, and it is necessary to further determine the vehicle head detection frame that has the correct association relationship with the license plate detection frame.

[0119] All vehicle front detection frames associated with the same license plate detection frame are obtained, and the obtained vehicle front detection frames are input into the intra-class occlusion relationship model. The intra-class occlusion relationship model compares the lowest edge lines of all vehicle front detection frames associated with the same license plate detection frame. Since the vehicle front detection frame is a rectangular frame, the lowest edge line of the vehicle front detection frame is the edge line of the vehicle front detection frame closest to the lower edge line of the current frame image.

[0120] S2424: Disconnecting the association between the vehicle head detection frame and the license plate detection frame to which the lowest edge line that is not closest to the lower edge line of the current frame image belongs.

[0121] Specifically, the vehicle front detection frame to which the lowest edge of the vehicle front detection frame is closest to the lower edge of the current frame image belongs is selected for matching with the license plate detection frame. It is determined that the vehicle front detection frame to which the lowest edge is closest to the lower edge of the current frame image belongs and the license plate detection frame correspond to the same target vehicle. It is determined that the vehicle front detection frame to which the lowest edge that is not closest to the lower edge of the current frame image belongs and the license plate detection frame correspond to different target vehicles, and the association relationship between the vehicle front detection frame and the license plate detection frame corresponding to different target vehicles is released.

[0122] The license plate detection frame and vehicle front detection frame that are associated are marked with the same ID.

[0123] S243: Associating each vehicle rear detection frame in the vehicle rear image set with each vehicle body detection frame in the vehicle body image set.

[0124] Specifically, the specific steps of associating the vehicle rear detection frame associated with the license plate detection frame in the above step S241 with each vehicle body detection frame in the vehicle body image set are as follows.

[0125] like Fig.10 As shown, Fig.10 yes Figure 6 A flowchart of a specific embodiment of step S243 in the vehicle tracking method is provided.

[0126] S2431: Calculate and obtain the overlap ratio between the rear end detection frame and each body detection frame in the body image set.

[0127] Specifically, the IOUs corresponding to each rear detection frame associated with the license plate detection frame and each body detection frame in the body image set are calculated, and then the overlap rate between each rear detection frame and each body detection frame is determined. In order to determine whether the rear detection frame and the body detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the third preset overlap ratio. The third preset overlap ratio is the intersection-and-union ratio between the rear detection frame and the body detection frame of the same vehicle.

[0128] S2432: In response to the overlap ratio between the rear end detection frame and the body detection frame exceeding a third preset overlap ratio, associating the rear end detection frame and the body detection frame corresponding to the overlap ratio.

[0129] Specifically, in response to the intersection-and-union ratio between the rear vehicle detection frame and the body detection frame exceeding a third preset overlap rate, it is determined that the body detection frame corresponding to the intersection-and-union ratio and the rear vehicle detection frame belong to the same target vehicle, and the body detection frame corresponding to the intersection-and-union ratio is associated with the rear vehicle detection frame.

[0130] All rear vehicle detection frames associated with the license plate detection frame in the rear vehicle image set and all body detection frames in the body image set are traversed, and the rear vehicle detection frame corresponding to the same target vehicle is associated with the body detection frame.

[0131] The associated vehicle rear detection frame and vehicle body detection frame are marked with the same ID.

[0132] S244: Associating each vehicle head detection frame in the vehicle head image set with each unassociated vehicle body detection frame in the vehicle body image set.

[0133] Specifically, the specific steps of associating the vehicle front detection frame associated with the license plate detection frame in the above step S242 with each vehicle body detection frame in the vehicle body image set that is not associated with the vehicle rear detection frame are as follows.

[0134] like Fig.11 As shown, Fig.11 yes Figure 6 A flowchart of a specific embodiment of step S244 in the vehicle tracking method is provided.

[0135] S2441: In response to the overlap ratio between the rear vehicle detection frame and the vehicle body detection frame not exceeding a third preset overlap ratio, calculating the overlap ratio between the unassociated vehicle body detection frame and each vehicle front detection frame.

[0136] Specifically, if the overlap rate between the rear detection frame and the body detection frame does not exceed the third preset overlap rate, it is determined that the rear detection frame and the body detection frame corresponding to the overlap rate belong to different target vehicles. In order to match the body detection frame to the corresponding target vehicle, the IOUs corresponding to each front detection frame associated with the license plate detection frame and the body detection frame in the body image set that is not associated with the rear detection frame are calculated, and then the overlap rate between the front detection frame and each body detection frame is determined. In order to determine whether the front detection frame and the body detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the fourth preset overlap rate. Among them, the fourth preset overlap rate is the intersection-and-union ratio between the front detection frame and the body detection frame of the same vehicle.

[0137] S2442: In response to the overlap ratio between the unassociated vehicle body detection frame and the vehicle front detection frame exceeding a fourth preset overlap ratio, associating the vehicle body detection frame and the vehicle front detection frame corresponding to the overlap ratio.

[0138] Specifically, in response to the intersection-and-joint ratio between the vehicle body detection frame and the vehicle front detection frame exceeding a fourth preset overlap rate, it is determined that the vehicle body detection frame and the vehicle front detection frame corresponding to the intersection-and-joint ratio belong to the same target vehicle, and the vehicle body detection frame and the vehicle front detection frame corresponding to the intersection-and-joint ratio are associated.

[0139] All vehicle front detection frames in the vehicle front image set that are associated with the license plate detection frame and all vehicle body detection frames in the vehicle body image set that are not associated with the vehicle rear detection frame are traversed, and the vehicle front detection frame and the vehicle body detection frame corresponding to the same target vehicle are associated.

[0140] The associated vehicle front detection frame and vehicle body detection frame are marked with the same ID.

[0141] S245: Associating each license plate detection frame in the license plate image set with each vehicle body detection frame in the vehicle body image set.

[0142] Specifically, in order to avoid missing detection of parts such as the rear and front of the vehicle, which affects the establishment of associations between other parts of the same target vehicle, the specific steps of associating each license plate detection frame in the license plate image set with each body detection frame in the body image set are as follows.

[0143] like Fig.12 As shown, Fig.12 yes Figure 6 A flowchart of a specific embodiment of step S245 in the vehicle tracking method is provided.

[0144] S2451: In response to the overlap ratio between the vehicle body detection frame and the vehicle front detection frame not exceeding a fourth preset overlap ratio, the overlap ratio between the vehicle body detection frame and each license plate detection frame is calculated.

[0145] Specifically, in response to the overlap rate between the unassociated vehicle body detection frame and the vehicle head detection frame not exceeding a fourth preset overlap rate, it is determined that the vehicle body detection frame and the license plate detection frame corresponding to the intersection-and-union ratio belong to different target vehicles.

[0146] The IOUs corresponding to the license plate detection frame and the body detection frame in the body image set are calculated, and then the overlap ratio between each license plate detection frame and each body detection frame is determined. In order to determine whether the license plate detection frame and the body detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the fifth preset overlap ratio. The fifth preset overlap ratio is the intersection-and-union ratio between the license plate detection frame and the body detection frame of the same vehicle.

[0147] S2452: In response to the overlap ratio between the unassociated vehicle body detection frame and the license plate detection frame exceeding a fifth preset overlap ratio, associating the unassociated vehicle body detection frame with the license plate detection frame.

[0148] Specifically, in response to the intersection-and-union ratio between the license plate detection frame and the body detection frame exceeding the fifth preset overlap rate, it is determined that the body detection frame corresponding to the intersection-and-union ratio and the license plate detection frame belong to the same target vehicle, and the body detection frame corresponding to the intersection-and-union ratio is associated with the license plate detection frame.

[0149] Traverse all license plate detection frames in the license plate image set and all body detection frames in the body image set, and associate the license plate detection frame and body detection frame corresponding to the same target vehicle.

[0150] The license plate detection frame and vehicle body detection frame that are associated are marked with the same ID.

[0151] S246: Associating each vehicle body detection frame in the vehicle body image set with each vehicle roof detection frame in the vehicle roof image set.

[0152] Specifically, the specific steps of associating the body detection frame associated with the rear vehicle detection frame in the above step S243, the body detection frame associated with the front vehicle detection frame in step S244, and the body detection frame associated with the license plate detection frame in step S245 with each roof detection frame in the roof image set are as follows.

[0153] like Fig.13 As shown, Fig.13 yes Figure 6 A flowchart of a specific embodiment of step S246 in the vehicle tracking method is provided.

[0154] S2461: Calculate and obtain the overlap ratios corresponding to the vehicle body detection frame and each vehicle roof detection frame in the vehicle roof image set.

[0155] Specifically, in order to establish a more accurate association relationship between the vehicle body detection frame and the vehicle roof detection frame, the overlap rate between the portion of the vehicle body detection frame close to the vehicle roof and the vehicle roof detection frame is calculated.

[0156] The IOUs corresponding to the part of the body detection frame near the roof and each roof detection frame are calculated, and then the overlap rate between each body detection frame and each roof detection frame is determined. In order to determine whether the body detection frame and the roof detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the sixth preset overlap rate. The sixth preset overlap rate is the intersection-and-union ratio between the roof detection frame and the body detection frame of the same vehicle.

[0157] In a specific embodiment, the corresponding IOU between the partial body detection frame close to the roof and the roof detection frame is calculated based on Formula 1.

[0158] IOU=(A∩BC) / (A∪B) (Formula 1)

[0159] In Formula 1, IOU represents the intersection-over-union ratio between the partial body detection frame close to the roof and the roof detection frame; A represents the partial body detection frame close to the roof, B represents the roof detection frame, and C represents the non-overlapping area between the roof detection frame and the body detection frame.

[0160] S2462: In response to the overlap ratio between the vehicle body detection frame and the vehicle roof detection frame exceeding a sixth preset overlap ratio, the vehicle body detection frame is associated with the vehicle roof detection frame.

[0161] Specifically, in response to the overlap rate between the partial vehicle body detection frame close to the vehicle roof and the vehicle roof detection frame exceeding a sixth preset overlap rate, the vehicle body detection frame is associated with the vehicle roof detection frame.

[0162] In one embodiment, in response to the intersection-and-joint ratio between a partial body detection frame close to the roof and the roof detection frame exceeding a sixth preset overlap rate, it is determined that the body detection frame corresponding to the intersection-and-joint ratio and the roof detection frame belong to the same target vehicle, and the body detection frame corresponding to the intersection-and-joint ratio is associated with the roof detection frame.

[0163] All body detection frames with associated relationships in the body image set and all roof detection frames in the roof image set are traversed, and the body detection frame and the roof detection frame corresponding to the same target vehicle are associated.

[0164] The roof detection frame and the body detection frame that are associated are marked with the same ID.

[0165] S247: Associating each unassociated vehicle roof detection frame in the vehicle roof image set with each vehicle rear detection frame in the vehicle rear image set.

[0166] Specifically, since the detection of the vehicle body cannot guarantee the output and output continuity, it is necessary to associate the roof detection frame that is not associated with the vehicle body detection frame with the rear detection frame so that all component detection frames corresponding to the same target vehicle can be associated with each other to ensure the integrity of the association results.

[0167] The specific steps of associating each unassociated vehicle roof detection frame in the vehicle roof image set with each vehicle rear detection frame in the vehicle rear image set are as follows.

[0168] like Fig.14 As shown, Fig.14 yes Figure 6 A flowchart of a specific embodiment of step S247 in the vehicle tracking method is provided.

[0169] S2471: In response to the overlap ratio between the vehicle body detection frame and the vehicle roof detection frame not exceeding a sixth preset overlap ratio, calculating an overlap ratio between an unassociated vehicle body detection frame and a vehicle rear detection frame.

[0170] Specifically, if the overlap rate between the body detection frame and the roof detection frame does not exceed the sixth preset overlap rate, it indicates that the body detection frame and the roof detection frame correspond to different target vehicles. In order to ensure the integrity of the association relationship, the roof detection frame that is not associated with the body detection frame needs to be matched with the rear detection frame or the body detection frame.

[0171] The IOUs corresponding to the roof detection frame and each rear detection frame that is not associated with the vehicle body detection frame are calculated, and then the overlap rate between each roof detection frame and each rear detection frame is determined. In order to determine whether the roof detection frame and the rear detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the seventh preset overlap rate. The seventh preset overlap rate is the intersection-and-union ratio between the roof detection frame and the rear detection frame of the same vehicle.

[0172] S2472: In response to the overlap ratio between the unassociated vehicle roof detection frame and the vehicle rear detection frame exceeding a seventh preset overlap ratio, the unassociated vehicle roof detection frame and the vehicle rear detection frame corresponding to the overlap ratio are associated.

[0173] In one embodiment, in response to the intersection-and-joint ratio between the rear vehicle detection frame and the unassociated roof detection frame exceeding the seventh preset overlap rate, it is determined that the rear vehicle detection frame and the roof detection frame corresponding to the intersection-and-joint ratio belong to the same target vehicle, and the rear vehicle detection frame and the roof detection frame corresponding to the intersection-and-joint ratio are associated.

[0174] All the roof detection frames in the roof image set that are not associated with the body detection frame and all the rear detection frames in the rear image set are traversed, and the rear detection frame corresponding to the same target vehicle is associated with the roof detection frame.

[0175] The associated roof detection frame and rear detection frame are marked with the same ID.

[0176] S248: Associating each unassociated vehicle roof detection frame in the vehicle roof image set with each vehicle head detection frame in the vehicle head image set.

[0177] Specifically, since the detection of the vehicle body and the rear of the vehicle cannot guarantee output and output continuity, it is necessary to associate the roof detection frame that is not associated with the vehicle body detection frame or the rear detection frame with the front detection frame, so that all component detection frames corresponding to the same target vehicle can be associated with each other to ensure the integrity of the association results.

[0178] The specific steps of associating each unassociated vehicle roof detection frame in the vehicle roof image set with each vehicle head detection frame in the vehicle head image set are as follows.

[0179] like Fig.15 As shown, Fig.15 yes Figure 6 A flowchart of a specific embodiment of step S248 in the vehicle tracking method is provided.

[0180] S2481: In response to the overlap ratio between the unassociated vehicle roof detection frame and the vehicle rear detection frame not exceeding a seventh preset overlap ratio, calculating the overlap ratio between the unassociated vehicle roof detection frame and the vehicle front detection frame.

[0181] Specifically, if the overlap rate between the rear detection frame and the roof detection frame does not exceed the seventh preset overlap rate, it indicates that the rear detection frame and the roof detection frame correspond to different target vehicles. In order to ensure the integrity of the association relationship, the roof detection frame that is not associated with the body detection frame needs to be matched with the rear detection frame or the body detection frame.

[0182] The IOUs corresponding to the roof detection frame that is not associated with the vehicle body detection frame and each vehicle head detection frame are calculated, and then the overlap rate between each roof detection frame and each vehicle head detection frame is determined. In order to determine whether the roof detection frame and the vehicle head detection frame correspond to the same target vehicle, the calculated intersection-and-union ratio is compared with the eighth preset overlap rate. The eighth preset overlap rate is the intersection-and-union ratio between the roof detection frame and the vehicle head detection frame of the same vehicle.

[0183] S2482: In response to the overlap ratio between the unassociated vehicle body detection frame and the vehicle front detection frame exceeding an eighth preset overlap ratio, the unassociated vehicle body detection frame corresponding to the overlap ratio is associated with the vehicle front detection frame.

[0184] In one embodiment, in response to the intersection-and-joint ratio between the front detection frame and the unassociated roof detection frame exceeding an eighth preset overlap rate, it is determined that the front detection frame and the roof detection frame corresponding to the intersection-and-joint ratio belong to the same target vehicle, and the front detection frame and the roof detection frame corresponding to the intersection-and-joint ratio are associated.

[0185] Traverse all the roof detection frames in the roof image set that are not associated with the body detection frame and the rear detection frame and all the front detection frames in the front image set, and associate the front detection frame corresponding to the same target vehicle with the roof detection frame.

[0186] The associated vehicle front detection frame and vehicle roof detection frame are marked with the same ID.

[0187] Since the target vehicle is in a densely occluded scene, the setting height of the image acquisition device exceeds the height of the vehicle, and the probability of the vehicle's roof being blocked by other objects is relatively small. Therefore, the position information of each target vehicle in the current frame image can be determined based on the position information of the target vehicle's roof detection frame.

[0188] like Fig.16 As shown, Fig.16 Step S24 is Figure 5 The detection result of the current frame image in is obtained by associating the component detection frames corresponding to the same target vehicle, so that different component detection frames corresponding to the same target vehicle correspond to the same ID.

[0189] S25: In response to the target vehicle not detecting the corresponding roof detection frame, a roof detection frame of the target vehicle in the current frame image is generated based on at least one corresponding position information of the body detection frame, the front detection frame, the rear detection frame and the license plate detection frame according to the positional relationship between the body, the front, the rear and the license plate and the roof, respectively.

[0190] Specifically, in order to be compatible with the tracking of all vehicles, some special roof parts may have the problem of poor detection stability, so some target vehicles may not be able to detect the roof detection frame of the target vehicle in the current frame. Since the body of the vehicle is large, the body detection frame can usually be detected and output. According to the positional relationship between the body, front, rear, license plate and the roof, the roof detection frame corresponding to the body detection frame in the current frame image can be generated based on the fusion of the body detection frame and the rear detection frame, front detection frame and / or license plate detection frame associated with the body detection frame.

[0191] In one embodiment, in response to the target vehicle not detecting a corresponding body detection frame, a body detection frame of the target vehicle is generated based on at least one of the roof detection frame, the front detection frame, the rear detection frame and the license plate detection frame according to the positional relationship between the roof, the front, the rear and the license plate and the body, respectively.

[0192] Specifically, in response to the target vehicle detecting the corresponding roof detection frame, the position information of the target vehicle in the current frame image can be determined based on the roof detection frame corresponding to the target vehicle.

[0193] In order to avoid the vehicle missing the body detection frame, that is, the target vehicle does not detect the corresponding body detection frame, the position information of the body detection frame can be determined based on the position relationship between the roof, front, rear, license plate and the body, based on the roof detection frame, front detection frame, rear detection frame and / or license plate detection frame with associated relationship. In other words, the body detection frame with occlusion and no output can be generated based on the position information fusion of the roof detection frame, front detection frame, rear detection frame and / or license plate detection frame, and the stable component detection frame is compatible with various types of vehicles for continuous tracking.

[0194] The above method can generate a roof detection frame or a body detection frame that has not been detected based on the fusion of the detected component detection frames, so that the overlap between vehicle components is significantly reduced during vehicle tracking, and the tracking continuity when there is occlusion between target vehicles can be significantly improved.

[0195] S26: predicting a roof prediction frame of the target vehicle in the current frame image according to the position information of the roof detection frame of each target vehicle in the historical frame image before the current frame image.

[0196] Specifically, in order to determine the correspondence between each target vehicle in the current frame image and each target vehicle corresponding to the historical frame image before the current frame image, the position area of ​​each target vehicle in the current frame image is predicted based on the position relationship of each target vehicle in the historical frame image.

[0197] The Kalman filter algorithm is used to predict the roof prediction box corresponding to the target vehicle in the current frame image based on factors such as the current posture and speed change of the roof detection box of the target vehicle in the historical frame image before the current frame image.

[0198] S27: Determine whether the target vehicle corresponding to the roof prediction frame is the same as the target vehicle corresponding to the roof detection frame based on the matching degree between the roof prediction frame and each roof detection frame in the current frame image.

[0199] like Fig.17 As shown, Fig.17 is the detection result of the current frame image in the fourth embodiment. Specifically, the roof detection frame and the roof prediction frame in the current frame image are calculated by the following method to calculate the loss value of the roof detection frame and the roof prediction frame.

[0200] like Fig.18 As shown, Fig.18 yes Figure 2 A flowchart of a specific embodiment of step S27 in the vehicle tracking method is provided.

[0201] S271: Calculate the center loss value between the center point position of the roof prediction frame and the center point position of the roof detection frame.

[0202] Specifically, the center loss value between the center point position of the vehicle roof prediction frame and the center point positions of all vehicle roof detection frames within a preset area of ​​the vehicle roof prediction frame is calculated. The preset area of ​​the vehicle prediction frame is a region range with a preset distance from the center point position of the vehicle roof prediction frame.

[0203] In a specific embodiment, the center loss values ​​S corresponding to the center point position of the roof prediction frame and the center point positions of each roof detection frame in the preset area are calculated based on Formula 2. point .

[0204]

[0205] In formula 2: S point It is the center loss value between the center point position of the roof prediction box and the center point position of the roof detection box; Indicates the coordinate value of the center point of the roof detection frame on the X-axis; Indicates the coordinate value of the center point of the roof prediction box on the X-axis; Indicates the coordinate value of the center point of the roof detection frame on the Y axis; Represents the coordinate value of the center point of the roof prediction box on the Y axis; ɑ and β are hyper parameters; ɑ = 1.5; β = 1.0.

[0206] S272: Calculate the width and height loss value between the width and height information of the roof prediction frame and the width and height information of the roof detection frame.

[0207] Specifically, the width and height loss value S between the width and height information of the roof prediction frame and the width and height information of the roof detection frame is calculated based on Formula 3: area .

[0208] S area (t pre ,d i )=sqrt(SQR_DIST((d width -t width ), (d height -t height ))) (Formula 3)

[0209] In formula 3: S area is the width and height loss value between the width and height information of the roof prediction box and the width and height information of the roof detection box; d width Indicates the width of the roof detection frame, t width Indicates the width of the roof prediction box, d height Indicates the width of the roof frame predicted by the tracking trajectory in the current frame, t height Indicates the height of the roof prediction box.

[0210] S273: Based on at least one of the center loss value and the width and height loss value, determine the loss value between the roof detection frame and the roof prediction frame.

[0211] Specifically, based on the weighted sum of the center loss value and the width and height loss value, the loss value between the roof detection box and the roof prediction box is determined.

[0212] In a specific embodiment, the loss value between the vehicle roof detection frame and the vehicle roof prediction frame is calculated based on Formula 4 and Formula 5.

[0213] s norm =sqrt(SQR_DIST(image_width / 2,image_height / 2)) (Formula 4)

[0214]

[0215] In the formula: S score is the loss value between the roof detection frame and the roof prediction frame;

[0216] SQR_DIST(x,y)=x 2 +y2 Represents the sum of squares of two variables; S area is the width and height loss value between the width and height information of the roof prediction box and the width and height information of the roof detection box; S point It is the center loss value between the center point position of the roof prediction box and the center point position of the roof detection box; and ε are hyperparameters; ε=0.8.

[0217] S274: In response to the loss value between the roof detection frame and the roof prediction frame being less than the loss value threshold, it is determined that the roof detection frame and the roof prediction frame corresponding to the loss value correspond to the same target vehicle.

[0218] Specifically, if the loss value between the vehicle detection frame and the roof prediction frame is less than the loss value threshold, it is determined that the roof detection frame corresponding to the loss value and the roof prediction frame correspond to the same target vehicle.

[0219] When the target vehicle enters the parking space, the front and rear of the vehicle are blocked, and the stability of the roof detection frame will be significantly reduced. False detection, missed detection and unstable output of the roof detection frame can easily lead to ID concatenation between target vehicles. Fig.19 As shown, Fig.19 is the detection result of the current frame image in the fifth embodiment. The car and the truck in the figure correspond to their own roof detection frames and body detection frames. The detection of the roof detection frame of the truck is unstable. During the matching process, the roof detection frame of the car is too large and is easy to form the best match with the body detection frame of the truck and the roof detection frame of the car, resulting in the truck body detection frame and the roof detection frame of the car corresponding to the same ID.

[0220] In order to avoid the situation where the roof detection frame of a target vehicle and the body detection frame of another target vehicle correspond to the ID of the same target vehicle, the body detection frame corresponding to the roof detection frame is selected, and the intersection and union ratio of the body detection frame and the other roof detection frames is calculated. In response to the intersection and union ratio of the body detection frame and the roof detection frame of another target vehicle exceeding the set value, the distance between the lowest edge line of the selected body detection frame and the lowest edge line of the body detection frame associated with the roof detection frame of another target vehicle is calculated. In response to the distance between the lowest edge line of the selected body detection frame and the lowest edge line of the body detection frame associated with the roof detection frame of another target vehicle exceeding the preset distance, whether the body detection frame of the target vehicle in the current frame image has an associated relationship with the roof detection frame is determined by the similarity between the body detection frame of the target vehicle in the current frame image and the body detection frame in the historical frame image.

[0221] In order to further determine whether the body detection frame of the target vehicle corresponding to the roof detection frame in the current frame image corresponds to the same target vehicle, it is determined whether the roof detection frame of the target vehicle matches the body detection frame based on the body image of the same target vehicle detected in the current frame image and the body image in the historical frame image.

[0222] S28: Based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image, determine whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct.

[0223] Specifically, in order to avoid further verification of whether the body detection frame and the roof detection frame with an associated relationship correspond to the same target vehicle, the body detection frame of the target vehicle in the current frame image and the body detection frame in the historical frame image are compared to determine whether the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is correct.

[0224] The specific steps of determining the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image are as follows.

[0225] like Fig. 20 As shown, Fig. 20 yes Figure 2 A flowchart of a specific embodiment of step S28 in the vehicle tracking method is provided.

[0226] S281: Calculate the body color similarity, body texture similarity, and body width and height similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle.

[0227] Specifically, the color histogram information of the body detection frame associated with the roof detection frame of the same target vehicle in the current frame image and the color histogram information of the body detection frame associated with the historical frame image are calculated. In a specific embodiment, L1 normalization is used to obtain a 25-bin color histogram of each channel of the body image area in the body detection frame. The body image corresponding to each body detection frame obtains a 75-dimensional vector Where n=75.

[0228] In a specific embodiment, the body color similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle is calculated based on Formula 6.

[0229]

[0230] In the formula: Scolor It is the body color similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle.

[0231] In one embodiment, texture features and texture similarity are calculated for 8 directions of each color channel of each region.

[0232] The texture features of the roof detection frame of the same target vehicle in the eight directions of each color channel of the body detection frame associated in the current frame image and the texture features of the body detection frame associated in the historical frame image are calculated, and the body texture similarity is determined based on the texture features between the body detection frames. Each body detection frame obtains the corresponding texture feature vector

[0233] In a specific embodiment, the body texture similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle is calculated based on Formula 7.

[0234]

[0235] In the formula: S texture It is the body texture similarity between the body detection box in the current frame image and the body detection box in the historical frame image of the same target vehicle.

[0236] In one embodiment, based on the width and height information of the body detection frame with an associated relationship in the current frame image and the width and height information of the body detection frame with an associated relationship in the historical frame image of the same target vehicle, the vehicle width and height similarity between the roof detection frame of the same target vehicle, the body detection frame with an associated relationship in the current frame image and the body detection frame with an associated relationship in the historical frame image is calculated.

[0237] In a specific embodiment, the vehicle width and height similarity between the vehicle body detection frame in the current frame image and the vehicle body detection frame in the historical frame image of the same target vehicle is calculated based on Formula 8 and Formula 9.

[0238] S area (t pre ,D i )=sqrt(SQR_DIST(α(d width -t width ),β(d height -t height ))) (Formula 8)

[0239] s score (t pre ,d i )=S area (tpre ,d i ) / s norm (Formula 9)

[0240] In the formula: S area It is the similarity of vehicle width and height between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle.

[0241] S282: Determine a similarity evaluation value of a body detection frame in a current frame image and a body detection frame in a historical frame image of the same target vehicle based on at least one of body color similarity, body texture similarity, and body width and height similarity.

[0242] Specifically, based on the weighted sum of the body color similarity, the body texture similarity and the body width and height similarity, the similarity evaluation value of the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image is determined.

[0243] In a specific embodiment, the similarity evaluation value of the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image is calculated based on Formula 10.

[0244] TConf i+1 =a*s color (r i ,r j )+b*s texture (r i ,r j )+c*S area (t pre ,d i ) (Formula 10)

[0245] In the formula: TConf i+1 is the similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle; a, b, and c are weighted values ​​respectively.

[0246] S283: In response to the similarity evaluation value exceeding the similarity evaluation value threshold, determining that the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is correct.

[0247] Specifically, if the similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle exceeds the similarity evaluation value threshold, it is determined that the body detection frame corresponding to the detected target vehicle in the current frame image and the body detection frame corresponding to the historical frame image both correspond to the same target vehicle, further verifying that the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is correct.

[0248] S284: In response to the similarity evaluation value not exceeding the similarity evaluation value threshold, the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is released.

[0249] Specifically, if the similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle does not exceed the similarity evaluation value threshold, it is determined that the body detection frame corresponding to the detected target vehicle in the current frame image and the body detection frame corresponding to the historical frame image correspond to different target vehicles, further verifying that the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is incorrect. In other words, the association relationship between the corresponding body detection frame and the roof detection frame in the current frame image is an invalid match. It is more likely to cause a string ID problem. At this time, the previously calculated association loss is set to an invalid flag in the matching result, and the matching is subsequently suppressed through this flag.

[0250] The association between the vehicle body detection frame corresponding to the similarity evaluation value and the vehicle roof detection frame in the current frame image is released.

[0251] The above method can avoid the ID of the overlapping and occluded target vehicle from moving up and down on the component detection frames of different target vehicles, so as to better maintain the continuity of the trajectory.

[0252] S29: Determine the position information of the target vehicle in the current frame image according to the vehicle body detection frame associated with the roof detection frame corresponding to the target vehicle.

[0253] Specifically, since the roof of the target vehicle is at the top of the vehicle and is not easily blocked, that is, the position information of the target vehicle in the current frame image can be determined based on the position information of the roof detection frame of the target vehicle. When the roof detection frame of the target vehicle is detected, the position information of the target vehicle is determined based on the position information of the roof detection frame of the target vehicle in the current frame image; when the roof detection frame of the target vehicle is not detected, the roof detection frame of the target vehicle can be predicted based on the fusion of other component detection frames of the target vehicle, and then the position information of the target vehicle in the current frame image is determined based on the predicted roof detection frame.

[0254] In another embodiment, since the body position of the target vehicle can better represent the actual position of the target vehicle, the position information of the target vehicle in the current frame image can be determined based on the body detection frame of the detected target vehicle or the body detection frame generated based on the detection frames of other components.

[0255] In one embodiment, the position information of the target vehicle in the current frame image is updated to the target vehicle's trajectory information set. Specifically, if the position information of the target vehicle is detected in the current frame image, the position information in the current frame image and the component detection frames of the target vehicle are updated to the target vehicle's trajectory information set.

[0256] In this embodiment, the vehicle tracking method includes acquiring a current frame image; the current frame image contains at least one target vehicle; performing vehicle part detection on the current frame image to determine vehicle part detection information of at least one target vehicle; predicting vehicle part prediction information of the target vehicle in the current frame image based on the position information of the target vehicle in the historical frame image before the current frame image; determining vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of the target vehicle in the current frame image; and determining the position information of the target vehicle in the current frame image based on the vehicle part detection information corresponding to the target vehicle. The present application performs vehicle part detection on the acquired current frame image to obtain unobstructed vehicle part detection information, determines the corresponding vehicle part detection information in the current frame image based on the position information of each target vehicle in the historical frame image before the current frame image, and then determines the position information of the target vehicle in the current frame image based on the determined vehicle part detection information, thereby avoiding the stability and continuity of target vehicle tracking being affected by the failure to detect position information due to partial obstruction of the target vehicle.

[0257] See also Fig.21 , Fig.21 This is a schematic block diagram of an embodiment of a vehicle tracking device provided by the present invention. This embodiment provides a vehicle tracking device 60 , which includes a detection module 61 , a prediction module 62 , a matching module 63 and an output module 64 .

[0258] The detection module 61 is used to perform vehicle part detection on the current frame image in the video to be detected, and determine vehicle part detection information of at least one target vehicle contained in the current frame image.

[0259] The prediction module 62 is used to predict the vehicle position prediction information of each target vehicle in the current frame image according to the position information of each target vehicle in the historical frame image of at least one target vehicle, and the historical frame image includes the video frame before the current frame image in the video to be detected.

[0260] The matching module 63 is used to determine the vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image.

[0261] The output module 64 is used to determine the position information of each target vehicle in the current frame image based on the vehicle part detection information corresponding to each target vehicle.

[0262] In this embodiment, vehicle part detection is performed on the acquired current frame image to obtain unobstructed vehicle part detection information, and the corresponding vehicle part detection information in the current frame image is determined based on the position information of each target vehicle in the historical frame image before the current frame image. Then, the position information of the target vehicle in the current frame image is determined according to the determined vehicle part detection information, so as to avoid the stability and continuity of target vehicle tracking being affected by the failure to detect the position information due to partial occlusion of the target vehicle.

[0263] See also Fig. 22 , Fig. 22 The terminal 70 in this embodiment includes: a processor 71, a memory 72, and a computer program stored in the memory 72 and executable on the processor 71. When the computer program is executed by the processor 71, the vehicle tracking method described above is implemented. To avoid repetition, it is not described one by one here.

[0264] See also Fig.23 , Fig.23 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention.

[0265] A computer-readable storage medium 90 is also provided in an embodiment of the present application. The computer-readable storage medium 90 stores a computer program 901. The computer program 901 includes program instructions. The processor executes the program instructions to implement the vehicle tracking method provided in the embodiment of the present application.

[0266] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0267] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0268] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0269] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0270] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0271] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0272] The above are only implementation modes of the present invention, and are not intended to limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A vehicle tracking method, characterized in that: The vehicle tracking method comprises: Performing vehicle part detection on a current frame image in the video to be detected, and determining vehicle part detection information of at least one target vehicle contained in the current frame image; Predicting vehicle position prediction information of each target vehicle in the current frame image according to position information of each target vehicle in the at least one target vehicle in the historical frame image, wherein the historical frame image includes a video frame before the current frame image in the video to be detected; Based on the vehicle part prediction information of each target vehicle in the current frame image, determining the vehicle part detection information corresponding to each target vehicle in the current frame image; Based on the vehicle part detection information corresponding to each target vehicle, determining the position information of each target vehicle in the current frame image; Wherein, the vehicle part detection information includes roof information; The performing vehicle part detection on the current frame image in the video to be detected and determining vehicle part detection information of at least one target vehicle contained in the current frame image includes: Performing vehicle multi-part detection on the current frame image in the video to be detected, and obtaining at least one component category image set corresponding to at least one target vehicle contained in the current frame image, wherein the component category image set includes multiple component images of the same category; Determining roof information of each of the target vehicles based on the at least one component category image set; The component category image set includes at least one of a roof image set, a body image set, a front image set, a rear image set, and a license plate image set; the body image set includes at least one body detection frame, The determining of the roof information of each of the target vehicles based on the at least one component category image set includes: Associating the component images contained in different component image sets included in the component category image set, and determining that the mutually associated component images correspond to the same target vehicle; determining roof information of the target vehicle based on the component image corresponding to the target vehicle; The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: In response to the target vehicle not detecting a corresponding body detection frame, generating a body detection frame of the target vehicle based on at least one of a roof detection frame, a front detection frame, a rear detection frame, and a license plate detection frame according to a positional relationship between different parts of the vehicle; Based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image, determine whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct.

2. The vehicle tracking method according to claim 1, characterized in that: The determining of the roof information of each of the target vehicles based on the at least one component category image set includes: Based on the similarity between the component images in the same component category image set and the confidence of the component images, the component images in the component category image set are screened; Based on the filtered at least one component category image set, roof information of each target vehicle is determined.

3. The vehicle tracking method according to claim 1, characterized in that: The license plate image set includes at least one license plate detection frame, and the vehicle rear image set includes at least one vehicle rear detection frame; The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: Calculate the overlap rate between each license plate detection frame in the license plate image set and each vehicle rear detection frame in the vehicle rear image set; In response to the overlap rate between the license plate detection frame and the vehicle rear detection frame exceeding a first preset overlap rate, the license plate detection frame and the vehicle rear detection frame corresponding to the overlap rate are associated.

4. The vehicle tracking method according to claim 3, characterized in that: The step of associating the component images contained in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle further includes: In response to the overlap rates respectively corresponding to a license plate detection frame and a plurality of vehicle rear detection frames exceeding the first preset overlap rate, the lowest edges of the plurality of vehicle rear detection frames associated with the license plate detection frame are compared; The association relationship between the vehicle rear detection frame and the license plate detection frame to which the lowest edge line that is not closest to the lower edge line of the current frame image belongs is released.

5. The vehicle tracking method according to claim 3, characterized in that: The vehicle front image set includes at least one vehicle front detection frame; The step of associating the component images contained in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle further includes: In response to the overlap rate between the license plate detection frame and the vehicle rear detection frame being lower than the first preset overlap rate, calculating the overlap rate between the unassociated license plate detection frame and each of the vehicle front detection frames in the vehicle front image set; In response to the overlap rate between the license plate detection frame and the vehicle front detection frame exceeding a second preset overlap rate, the license plate detection frame and the vehicle front detection frame corresponding to the overlap rate are associated.

6. The vehicle tracking method according to claim 5, characterized in that: The step of associating the component images contained in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle further includes: In response to the overlap rates respectively corresponding to a license plate detection frame and a plurality of vehicle front detection frames all exceeding the second preset overlap rate, the lowest edges of the plurality of vehicle front detection frames associated with the license plate detection frame are compared; The association relationship between the vehicle head detection frame and the license plate detection frame to which the lowest edge line that is not closest to the lower edge line of the current frame image belongs is released.

7. The vehicle tracking method according to any one of claims 1 to 6, characterized in that: The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: Calculate the overlap rate between the vehicle rear detection frame and each of the vehicle body detection frames in the vehicle body image set; In response to the overlap rate between the vehicle rear detection frame and the vehicle body detection frame exceeding a third preset overlap rate, the vehicle rear detection frame corresponding to the overlap rate is associated with the vehicle body detection frame.

8. The vehicle tracking method according to claim 7, characterized in that: The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: In response to the overlap rate between the vehicle rear detection frame and the vehicle body detection frame not exceeding the third preset overlap rate, calculating the overlap rate between the unassociated vehicle body detection frame and each vehicle front detection frame; In response to the overlap rate between the unassociated vehicle body detection frame and the vehicle front detection frame exceeding a fourth preset overlap rate, the vehicle body detection frame and the vehicle front detection frame corresponding to the overlap rate are associated.

9. The vehicle tracking method according to claim 8, characterized in that: The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: In response to the overlap rate between the unassociated vehicle body detection frame and the vehicle head detection frame not exceeding the fourth preset overlap rate, calculating the overlap rate between the unassociated vehicle body detection frame and each license plate detection frame; In response to the overlap rate between the unassociated vehicle body detection frame and the license plate detection frame exceeding a fifth preset overlap rate, the unassociated vehicle body detection frame is associated with the license plate detection frame.

10. The vehicle tracking method according to claim 1, characterized in that: The vehicle roof image set includes at least one vehicle roof detection frame, The step of associating the component images included in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle includes: Calculate the overlap ratios of the vehicle body detection frame and each of the vehicle roof detection frames in the vehicle roof image set; In response to the overlap rate between the vehicle body detection frame and the vehicle roof detection frame exceeding a sixth preset overlap rate, the vehicle body detection frame corresponding to the overlap rate is associated with the vehicle roof detection frame.

11. The vehicle tracking method according to claim 10, characterized in that: The calculating of the overlap ratios respectively corresponding to the vehicle body detection frame and each of the vehicle roof detection frames in the vehicle roof image set includes: Calculate the overlap rate between the vehicle body detection frame and the vehicle roof detection frame at a portion close to the vehicle roof; In response to the overlap rate between the vehicle body detection frame and the vehicle roof detection frame exceeding a sixth preset overlap rate, associating the vehicle body detection frame corresponding to the overlap rate with the vehicle roof detection frame, including: In response to the overlap rate between the portion of the vehicle body detection frame close to the vehicle roof and the vehicle roof detection frame exceeding the sixth preset overlap rate, the vehicle body detection frame is associated with the vehicle roof detection frame.

12. The vehicle tracking method according to claim 11, characterized in that: The determining whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image comprises: Calculate the body color similarity, body texture similarity and body width and height similarity between the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle; Determine a similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image for the same target vehicle based on at least one of the body color similarity, the body texture similarity, and the body width and height similarity; In response to the similarity evaluation value exceeding the similarity evaluation value threshold, it is determined that the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is correct.

13. The vehicle tracking method according to claim 12, characterized in that: The determining whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image comprises: In response to the similarity evaluation value not exceeding the similarity evaluation value threshold, the association relationship between the roof detection frame and the body detection frame corresponding to the target vehicle in the current frame image is released.

14. The vehicle tracking method according to claim 12, characterized in that: The determining, based on at least one of the vehicle body color similarity, the vehicle body texture similarity, and the vehicle body width and height similarity, a similarity evaluation value of the vehicle body detection frame in the current frame image and the vehicle body detection frame in the historical frame image of the same target vehicle comprises: Based on the weighted sum of the body color similarity, the body texture similarity and the body width and height similarity, a similarity evaluation value of the body detection frame in the current frame image and the body detection frame in the historical frame image of the same target vehicle is determined.

15. The vehicle tracking method according to claim 8, characterized in that: The step of associating the component images contained in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle further includes: In response to the overlap rate between the vehicle body detection frame and the vehicle roof detection frame not exceeding a sixth preset overlap rate, calculating an overlap rate between the unassociated vehicle roof detection frame and vehicle rear detection frame; In response to the overlap rate between the unassociated vehicle roof detection frame and the vehicle rear detection frame exceeding a seventh preset overlap rate, the unassociated vehicle roof detection frame corresponding to the overlap rate is associated with the vehicle rear detection frame.

16. The vehicle tracking method according to claim 15, characterized in that: The step of associating the component images contained in different component image sets included in the component category image set to determine that the mutually associated component images correspond to the same target vehicle further includes: In response to the overlap rate between the unassociated vehicle roof detection frame and the vehicle rear detection frame not exceeding the seventh preset overlap rate, calculating the overlap rate between the unassociated vehicle roof detection frame and the vehicle front detection frame; In response to the overlap rate between the unassociated vehicle roof detection frame and the vehicle front detection frame exceeding an eighth preset overlap rate, the unassociated vehicle roof detection frame corresponding to the overlap rate is associated with the vehicle front detection frame.

17. The vehicle tracking method according to claim 1, characterized in that: The roof information includes a roof detection frame; The determining of the roof information of the target vehicle based on the component image corresponding to the target vehicle includes: In response to the target vehicle not detecting a corresponding roof detection frame, a roof detection frame of the target vehicle in the current frame image is generated based on at least one corresponding position information of the body detection frame, the front detection frame, the rear detection frame and the license plate detection frame according to the positional relationship between the body, the front, the rear and the license plate and the roof, respectively.

18. The vehicle tracking method according to claim 1, characterized in that: The determining, based on the vehicle part prediction information of each target vehicle in the current frame image, the vehicle part detection information corresponding to each target vehicle in the current frame image, includes: According to the matching degree between the roof prediction frame of the target vehicle in the current frame image and each roof detection frame, it is determined whether the target vehicle corresponding to the roof prediction frame is the same as the target vehicle corresponding to the roof detection frame.

19. The vehicle tracking method according to claim 18, characterized in that: The determining, based on the matching degree between the roof prediction frame of the target vehicle in the current frame image and each roof detection frame, whether the target vehicle corresponding to the roof prediction frame is the same as the target vehicle corresponding to the roof detection frame comprises: Calculate the center loss value between the center point position of the roof prediction frame and the center point position of the roof detection frame; Calculating and obtaining a width and height loss value between the width and height information of the roof prediction frame and the width and height information of the roof detection frame; Determine a loss value between the vehicle roof detection frame and the vehicle roof prediction frame based on at least one of the center loss value and the width and height loss value; In response to the loss value between the roof detection frame and the roof prediction frame being less than a loss value threshold, it is determined that the roof detection frame corresponding to the loss value and the roof prediction frame correspond to the same target vehicle.

20. The vehicle tracking method according to claim 1, characterized in that: The determining, based on the vehicle part detection information corresponding to each target vehicle, the position information of each target vehicle in the current frame image includes: The position information of the target vehicle in the current frame image is determined according to the vehicle body detection frame associated with the roof detection frame corresponding to the target vehicle.

21. A vehicle tracking device, characterized in that: The vehicle tracking device comprises: A detection module is used to perform vehicle part detection on a current frame image in a video to be detected, and determine vehicle part detection information of at least one target vehicle contained in the current frame image; the vehicle part detection information includes roof information; and is also used to perform vehicle multi-part detection on the current frame image in the video to be detected, and obtain at least one component category image set corresponding to at least one target vehicle contained in the current frame image, wherein the component category image set includes multiple component images of the same category; based on the at least one component category image set, the roof information of each target vehicle is determined; the component category image set includes at least one of a roof image set, a body image set, a front image set, a rear image set, and a license plate image set; and different component images included in the component category image set are collected into The component images included are associated to determine that the mutually associated component images correspond to the same target vehicle; it is also used to determine the roof information of the target vehicle based on the component image corresponding to the target vehicle; in response to the target vehicle not detecting the corresponding body detection frame, based on the positional relationship between different parts of the vehicle, at least one of the roof detection frame, the front detection frame, the rear detection frame and the license plate detection frame is generated to detect the body detection frame of the target vehicle; based on the similarity between the body detection frame of the same target vehicle in the current frame image and the body detection frame in the historical frame image before the current frame image, it is determined whether the association between the roof detection frame corresponding to the target vehicle in the current frame image and the body detection frame is correct; A prediction module, used for predicting vehicle position prediction information of each target vehicle in the current frame image according to the position information of each target vehicle in the at least one target vehicle in the historical frame image, wherein the historical frame image includes a video frame before the current frame image in the video to be detected; A matching module, for determining the vehicle part detection information corresponding to each target vehicle in the current frame image based on the vehicle part prediction information of each target vehicle in the current frame image; The output module is used to determine the position information of each target vehicle in the current frame image based on the vehicle part detection information corresponding to each target vehicle.

22. A terminal, characterized in that: The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is used to execute program data to implement the steps in the vehicle tracking method according to any one of claims 1 to 20.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the vehicle tracking method according to any one of claims 1 to 20 are implemented.

Images