Tracking box maintaining method and device, electronic equipment and storage medium

Abstract

The invention discloses a tracking frame maintaining method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target vehicle with a missing tracking frame according to video data, and determining the motion state of the target vehicle; when the motion state is static, acquiring a first motion track of the target vehicle in the video data and a second motion track of surrounding vehicles corresponding to the target vehicle; determining a target front vehicle in the surrounding vehicles according to a distance comparison result of the first movement track and the second movement track; and determining that the motion state of the target front vehicle is static, determining that the target vehicle is in a queuing state, and taking a historical tracking frame of the target vehicle in the video data as a missing tracking frame of the target vehicle. According to the embodiment of the invention, the data of the lost tracking frame in the image tracking process of the motor vehicle can be repaired, the loss probability of the tracking target can be reduced, the accuracy of subsequent target detection can be improved, and the application scene of target tracking can be expanded.

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a tracking frame maintaining method, device, electronic device and storage medium. Background Art

[0002] In the field of smart transportation, video image-based recognition technology is widely used in vehicle detection, tracking, and identification of motor vehicles. On this basis, traffic intersection camera recognition can obtain important data such as motor vehicle traffic flow, vehicle type, and vehicle license plate through further analysis.

[0003] At present, most multi-target tracking adopts the Tracking-by-detection algorithm, and the tracking effect depends on the detection results of the vehicle target by the video detector. However, in the case of densely packed vehicles, the video detector may not be able to accurately capture the target vehicle due to the occlusion of the front vehicle, resulting in the loss of the tracking frame. When the tracking frame in the video frame processed by the video detector is lost, it is impossible to accurately count the traffic flow, resulting in the limitation of the application of video data. Therefore, a tracking frame maintenance method is urgently needed to reduce the probability of tracking frame loss in video data and enhance the integrity of video data. Summary of the invention

[0004] The present invention provides a tracking frame maintaining method, device, electronic device and storage medium, which can compensate for the data of the lost tracking frame during the target tracking process, reduce the probability of losing the tracked target, and help improve the accuracy of subsequent target detection.

[0005] According to one aspect of the present invention, a tracking frame maintaining method is provided, wherein the method comprises:

[0006] Acquire a target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle;

[0007] When the motion state is stationary, obtaining a first motion trajectory of the target vehicle and a second motion trajectory of surrounding vehicles corresponding to the target vehicle in the video data;

[0008] Determining a target preceding vehicle among the surrounding vehicles according to a distance comparison result between the first motion trajectory and the second motion trajectory;

[0009] It is determined that the motion state of the target front vehicle is stationary, and the target vehicle is in a queue state, and the historical tracking frame of the target vehicle in the video data is used as the missing tracking frame of the target vehicle.

[0010] According to another aspect of the present invention, a tracking frame maintaining device is provided, wherein the device comprises:

[0011] A missing detection module is used to obtain a target vehicle whose tracking frame is missing according to video data, and determine the motion state of the target vehicle;

[0012] A maintenance trigger module is used to obtain a first motion track of the target vehicle and a second motion track of surrounding vehicles corresponding to the target vehicle in the video data when the motion state is stationary;

[0013] A preceding vehicle determining module, configured to determine a target preceding vehicle among the surrounding vehicles according to a distance comparison result between the first motion trajectory and the second motion trajectory;

[0014] The tracking frame adding module is used to determine that the motion state of the target front vehicle is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the tracking frame maintaining method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tracking frame maintaining method according to any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention determines the target vehicle that has lost the tracking frame in the video data, obtains the motion state of the target vehicle, and when it is determined that the motion state of the target vehicle is stationary, determines the first motion trajectory of the target vehicle and the second motion trajectory of the surrounding vehicles of the target vehicle in the video data, obtains the distance comparison result by comparing the first motion trajectory and the second motion trajectory, and determines the target front vehicle of the target vehicle according to the distance comparison result, and when the motion state of the target front vehicle is stationary, obtains the historical tracking frame of the target vehicle in the video data, thereby compensating for the data of the lost tracking frame during the target tracking process, which can reduce the probability of losing the tracked target and help maintain the target tracking state.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a tracking frame maintaining method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a flowchart of another tracking frame maintaining method provided according to Embodiment 2 of the present invention;

[0025] Figure 3 is a flowchart of another tracking frame maintaining method provided according to Embodiment 3 of the present invention;

[0026] Figure 4 is a flowchart of another tracking frame maintaining method provided according to Embodiment 4 of the present invention;

[0027] Figure 5 This is an example diagram of a tracking frame loss caused by missed detection provided by Embodiment 5 of the present invention;

[0028] Figure 6 is an example diagram of a trajectory position relationship provided according to Embodiment 5 of the present invention;

[0029] Figure 7 is an example diagram of a trajectory position relationship provided according to Embodiment 5 of the present invention;

[0030] Figure 8 This is an example diagram of motion trajectory comparison provided according to Embodiment 5 of the present invention;

[0031] Fig. 9 This is an example diagram of a tracking frame maintenance effect provided according to Embodiment 5 of the present invention;

[0032] Fig.10 is a structural schematic diagram of a tracking frame maintaining device according to a sixth embodiment of the present invention;

[0033] Fig.11 It is a structural schematic diagram of an electronic device for implementing the tracking frame maintaining method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1 This is a flow chart of a tracking frame maintenance method provided according to the first embodiment of the present invention. This embodiment is applicable to the situation where the tracking state is interrupted due to the missing tracking frame when tracking multiple targets in a dense motor vehicle scene and the tracking frame is supplemented. The method can be executed by a tracking frame maintenance device, which can be implemented in the form of hardware and / or software, and can be configured in a server or terminal device. Figure 1 As shown, the method includes:

[0038] Step 110: Acquire the target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle.

[0039] In an embodiment of the present invention, the video data may be any traffic intersection video, and the target detection algorithm may include Faster R-CNN, YOLO, SSD, etc. The target detection algorithm may process each video frame in the video data, obtain the coordinates of all vehicle targets in each video frame, and send the tracking frames of multiple consecutive frames to the target tracking algorithm for processing to obtain the tracking status of all vehicles. Each vehicle target has a tracker, and the tracker will match the latest tracking frame of the vehicle target in each video frame. The latest coordinate position of the vehicle target can be updated through the latest tracking frame to achieve the effect of maintaining target tracking. The motion state of the target vehicle can be determined by the pixel speed of the target vehicle or the background frame difference detection, etc.

[0040] Specifically, the video data processed by the target detection algorithm can be detected, and the detection can include determining whether a tracking frame is configured for each motor vehicle in the video data, or, if the corresponding tracking frame is not obtained in the video frame of the video data by the tracker, if there is a motor vehicle without a tracking frame in the video data, the motor vehicle can be used as the target vehicle, and the motion state of the target vehicle can be detected through the video data. For example, the latest multiple video frames including the target vehicle can be extracted from the video data, and the pixel speed of the target vehicle can be determined through the multiple video frames to judge the motion state of the target vehicle. Alternatively, the frame difference method can be used to perform background detection on the multiple video frames to obtain the foreground in the video frame, and the motion state of the target vehicle can be determined based on the situation of the foreground in the video frame.

[0041] Step 120: When the motion state is stationary, obtain a first motion trajectory of the target vehicle in the video data and a second motion trajectory of surrounding vehicles corresponding to the target vehicle.

[0042] Among them, the first motion trajectory can be the motion trajectory of the target vehicle determined by the video data, and the first motion trajectory can be determined by the positions of at least two position points in space. The position point can be a position point on the target vehicle or the tracking frame of the target vehicle in the video data. For example, the above position point can include the bottom edge midpoint of the tracking frame of the target vehicle, and the coordinate position of each bottom edge midpoint in the video frame can be used as the position point constituting the first motion trajectory in the video data. Further, the position point in the first motion trajectory can be the latest position point from the current moment. For example, the first motion trajectory can include the bottom edge midpoint of the tracking frame of the target vehicle in the previous video frame and the previous two video frames of the current video frame in the video data. The surrounding vehicles can be motor vehicle video data determined in the video data. In the same video frame, the surrounding vehicles can be vehicles within a certain distance range from the target vehicle. The distance between the surrounding vehicles and the target vehicle can be determined by the pixel distance or the actual distance. The surrounding vehicles can be determined by the image frame in the video data where the tracking frame is missing, that is, they can be determined by the current video frame in the video data. The second motion trajectory may be the motion trajectory of each surrounding vehicle, and the second motion trajectory may be composed of multiple position points of each surrounding vehicle, and the position points may be corresponding surrounding vehicles located in the video data. It is understandable that the position points may at least include the bottom midpoint of the tracking frame of the surrounding vehicle or the intersection of the diagonal lines of the tracking frame. Further, in order to improve the accuracy of the tracking frame compensation, the distance between the position points in the second motion trajectory may be at least greater than the threshold distance, thereby improving the accuracy of the second motion trajectory and preventing positioning errors of the tracking frame of the target vehicle due to too dense position points.

[0043] In an embodiment of the present invention, when it is determined that the motion state of the target vehicle is stationary, at least two position points of the target vehicle can be determined in the video data, and the position points are obtained through the historical tracking frame trajectory of the target vehicle. For example, the bottom midpoints or diagonal intersections of two non-closely adjacent tracking frames of the historical tracking frame on the timeline can be taken to form a first motion trajectory. Other vehicles within a specified range can also be selected as surrounding vehicles in the video data with the target vehicle as the center. The range of selecting the target vehicle in the video data can at least include a video frame containing the target vehicle. The shorter the time interval between the video frame and the current video frame that determines the missing tracking frame, the higher the accuracy of judging that the target vehicle is in a queue state through the surrounding vehicles. For each surrounding vehicle obtained, the second motion trajectory of the surrounding vehicle can be extracted in the video data. For example, the video data includes a tracker of the surrounding vehicle. According to the tracker, the current video frame can be used as the end point to obtain other video frames in the past period of time. The position points of the surrounding vehicles are determined through the current video frame and other video frames, and the second motion trajectory can be formed through the position points.

[0044] Step 130: determine the target preceding vehicle among the surrounding vehicles according to the distance comparison result between the first motion trajectory and the second motion trajectory.

[0045] Wherein, the distance comparison result may include the comparison result of the distance position between the first motion trajectory and the second motion trajectory. The distance comparison between the first motion trajectory and the second motion trajectory may be determined by the shortest distance between any position points of the first motion trajectory and the second motion trajectory, or the shortest distance between the trajectory segments of the first motion trajectory and the second motion trajectory, or the shortest distance between any position point in the first motion trajectory and the trajectory segment of the second motion trajectory. The distance comparison result may determine the spatial position relationship between the target vehicle and the surrounding vehicles, so as to determine whether the target vehicle is in a queue state in combination with the motion state of the target vehicle and the surrounding vehicles, thereby facilitating the compensation of the missing tracking frame of the target vehicle. The target front vehicle may be the first vehicle located in front of the target vehicle. The target front vehicle may be determined by the distance comparison results of the first motion trajectory and the second motion trajectory. The distance comparison results of the first motion trajectory and the second motion trajectory may be used to determine a vehicle in the surrounding vehicles that is relatively stationary with the target vehicle and located in front of the target vehicle as the target front vehicle.

[0046] In an embodiment of the present invention, the first motion trajectory of the target vehicle can be compared with the second motion trajectories of different surrounding vehicles respectively. The distance comparison result between the second motion trajectory and the first motion trajectory can be used to determine the vehicle in the surrounding vehicles that is in a relatively stationary state with the target vehicle and is located in front of the target vehicle as the target front vehicle.

[0047] Exemplarily, by comparing the distances between the position points of the first motion trajectory and the second motion trajectory at the same time, the changing trends of these distances over time can be used as the distance comparison results, and the surrounding vehicles that are ahead of the target vehicle and whose changing trends have not changed can be used as the target front vehicle.

[0048] Step 140: Determine that the motion state of the target vehicle in front is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

[0049] Among them, the historical tracking box can be a tracking box marked for the target vehicle in a historical video frame in the video data. The historical tracking box can be a video frame or a few video frames before the current video frame where the tracking box is missing. The historical tracking box can include attribute parameters such as position attributes, size attributes, and category attributes. The attribute parameters can indicate relevant information of the marked target vehicle.

[0050] Specifically, the motion state of the target vehicle in front can be extracted through video data, for example, it can be determined by the pixel speed of the target vehicle in front or by background frame difference detection. When it is determined that the target vehicle in front is stationary, the target vehicle can be in a queue state, and one or more historical video frames can be searched in the video data. The historical tracking frame corresponding to the target vehicle can be extracted from the historical video frame, and the tracking frame can be generated for the target vehicle in the video data according to the historical tracking frame. It can be understood that the method of extracting the historical tracking frame corresponding to the target vehicle in the historical video frame can include that the target vehicle in the video data can have its corresponding tracker, and the historical tracking frame of the target vehicle can be extracted in the video data according to the tracker.

[0051] For example, the tracking frame is not missing in image frame A, but the tracking frame of the target vehicle is missing in image frame B. Since the target vehicle in front and the target vehicle are stationary, the positional relationship between the target vehicle in front and the target vehicle does not change, and the target vehicles are in a queue state, the tracking frame of the target vehicle can be selected as the historical tracking frame in image frame A. The missing tracking frame can be set for the target vehicle in the video data through the historical tracking frame, thereby compensating for the lost tracking frame in the video data.

[0052] According to an embodiment of the present invention, a target vehicle that has lost its tracking frame is determined in video data, the motion state of the target vehicle is obtained, and when the motion state of the target vehicle is determined to be stationary, a first motion trajectory of the target vehicle and a second motion trajectory of vehicles surrounding the target vehicle are determined in the video data, a distance comparison result is obtained by comparing the first motion trajectory and the second motion trajectory, and a target preceding vehicle of the target vehicle is determined according to the distance comparison result, and when the motion state of the target preceding vehicle is stationary, a historical tracking frame of the target vehicle is obtained in the video data, thereby compensating for the data of the lost tracking frame during the target tracking process, which can reduce the probability of loss of the tracked target and help maintain the target tracking state.

[0053] Further, based on the above-mentioned embodiment of the invention, determining the motion state of the target vehicle includes at least one of the following:

[0054] Determining the motion state as stationary based on the pixel velocity of the target vehicle;

[0055] The motion state is determined to be stationary based on the background frame difference of the target vehicle.

[0056] In an embodiment of the present invention, the motion state of the target vehicle can be determined by the pixel speed of the target vehicle, or by the background frame difference of the target vehicle. Specifically, the pixel speed can be the pixel distance moved by the target vehicle in the video data per unit time. The pixel speed can be expressed in pixels / second, which represents the number of pixels moved per second by the target vehicle on the image screen of the video data. It can be based on the positions of two different time points in the video data and measure the pixel distance between the two positions. The ratio of the pixel distance to the time interval between the two different time points can be used as the pixel speed. Whether the motion state of the target vehicle is stationary can be determined according to the value of the pixel speed. In some embodiments, the pixel speed can also be normalized to eliminate the speed inconsistency problem caused by "large near and small far".

[0057] In other embodiments of the invention, the motion state of the target vehicle can be determined by the background frame difference. Specifically, the motion state can be detected by detecting the difference between the current frame of the target vehicle and the background frame in the video data. The background frame can be the average value of the video frames over a period of time in the video data. The image of the target vehicle losing the tracking frame can be used as the current frame to determine the absolute difference or square difference between the current frame and the background frame. The motion state of the target vehicle can be determined by the value of the absolute difference or the square difference. Whether the motion state of the target vehicle is stationary.

[0058] In some embodiments of the invention, determining the motion state as stationary based on the pixel speed of the target vehicle includes: determining at least two frames of vehicle images of the target vehicle in the video data; determining the vertical pixel speed and the horizontal pixel speed of the target vehicle according to the vehicle images; and normalizing the vertical pixel speed and the horizontal pixel speed to obtain the pixel speed.

[0059] In the embodiment of the present invention, due to the problem of large near and small far away in the video data, when the target vehicle is located at a farther position in the image, the determined pixel speed is smaller. In order to reduce this influence, the pixel speed can be normalized in the process of calculating the pixel speed. The pixel speed in the vertical direction or the pixel speed in the horizontal direction can be divided by the tracking frame width to obtain a dimensionless normalized pixel speed, thereby improving the accuracy of motion state judgment.

[0060] Specifically, at least two frames of vehicle images of the target vehicle can be extracted from the video data, and the pixel speeds of the target vehicle in the vertical and horizontal directions can be determined according to the position of the target vehicle in the vehicle images and the time when the vehicle images are collected. The vertical pixel speed and the horizontal pixel speed can be normalized, and the normalization of the pixel speed can be achieved by setting the vertical pixel speed and the horizontal pixel speed to the width of the tracking frame respectively, and the normalized vertical pixel speed and horizontal pixel speed can be used as the pixel speed.

[0061] Embodiment 2

[0062] Figure 2 is a flowchart of another tracking frame maintenance method provided according to the second embodiment of the present invention. The embodiment of the present invention is a specific embodiment based on the above embodiment, and introduces the process of obtaining the first motion trajectory and the second motion trajectory. Figure 2 The method provided in the embodiment of the present invention specifically includes the following steps:

[0063] Step 210: Acquire the target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle.

[0064] Step 220: When the motion state is static, extract at least a first number of historical video frames closest to the current moment in the video data, and construct a first motion trajectory according to the vehicle position of the target vehicle in each historical video frame.

[0065] Among them, the first number can be a pre-configured number of video frame acquisitions, and the first number can determine the number of position points constituting the first motion trajectory. The vehicle position can be the position of the target vehicle in the historical video frame, and the position can be determined by the position coordinates of the target vehicle in the historical video frame. The position coordinates can be the midpoint of the bottom edge of the tracking frame of the target vehicle, the intersection of the diagonal of the tracking frame, etc. It is further not excluded that the position point of the first motion trajectory can correspond to the center point of the target vehicle in the historical video frame, the edge point of the door, the center point of the bumper, etc.

[0066] In an embodiment of the present invention, when it is determined that the motion state of the target vehicle is stationary, the video data can be based on the current moment, and the video frame closest to the current moment can be selected in the video data as the historical video frame. The number of the historical video frames can be a pre-configured first number, and the vehicle position of the vehicle can be extracted in the historical video frame. The vehicle position can be obtained by a tracker or detector of the target vehicle in the historical video frame of the video data. It can be understood that the method of obtaining the vehicle position does not exclude image recognition of the historical video frame, so as to determine the position of the target vehicle in the historical video frame.

[0067] Step 230: extract a trajectory set of each surrounding vehicle in the video data, and select trajectory points that satisfy at least a second number in the trajectory set to form a second motion trajectory, wherein a distance between two adjacent trajectory points is at least greater than a threshold distance.

[0068] Among them, the trajectory set can be a set of trajectory points generated by the video data through a pre-performed target tracking algorithm, and the trajectory points in the trajectory set can be determined by the tracking frame generated by the target tracking algorithm. The trajectory set can be attribute information of the video data. The second number can be the number of trajectory points constituting the second motion trajectory, and the second number can be pre-configured. The threshold distance can be the distance for screening the trajectory points in the second motion trajectory. When the distance between the trajectory points is greater than the threshold distance, the second motion trajectory can be prevented from being too dense and causing inaccurate selection of the target front vehicle. It can be understood that the historical tracking trajectories of all surrounding vehicles can be saved in the video data, but not every frame of the video frame saves the current tracking frame information. For a surrounding vehicle, a certain distance can be maintained between the tracking frames in the historical tracking trajectory. The distance is recorded as the threshold distance, and its value can be set according to the multiple of the longest side of the tracking frame. The historical tracking trajectory of the surrounding vehicles can be obtained by connecting the midpoints of the bottom edges of the tracking frames of each surrounding vehicle in the time sequence of the corresponding video frames.

[0069] In an embodiment of the present invention, a trajectory set of surrounding vehicles of the target vehicle can be extracted in the video data, and one or more trajectory points can be selected from the trajectory set to form a second motion trajectory. It can be understood that, among the selected trajectory points, the distance between any two adjacent trajectory points is greater than a threshold distance, thereby reducing the density of the trajectory points in the second motion trajectory.

[0070] Step 240: determine the target preceding vehicle among the surrounding vehicles according to the distance comparison result between the first motion trajectory and the second motion trajectory.

[0071] Step 250: Determine that the motion state of the target vehicle in front is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

[0072] In an embodiment of the present invention, a target vehicle that has lost its tracking frame is determined in video data, the motion state of the target vehicle is obtained, and when the motion state of the target vehicle is determined to be stationary, a first motion trajectory is determined in a first image frame corresponding to the target vehicle in the video data, and a second motion trajectory of surrounding vehicles corresponding to the target vehicle is extracted in the video data, and a comparison result is obtained by comparing the first motion trajectory and the second motion trajectory, and a target front vehicle of the target vehicle is determined in the surrounding vehicles according to the comparison result, and when the motion state of the target front vehicle is determined to be stationary, a target tracking frame of the target front vehicle is obtained in the video data, and a tracking frame lost by the target vehicle is generated based on the target tracking frame. The embodiment of the present invention can compensate for the missing tracking frame in the target tracking process, determine the positional relationship between vehicles by comparing the motion trajectories, and repair the video data of the missing tracking frame based on the stationary positional relationship, which can reduce the probability of losing the tracking target, help maintain the tracking state of the target vehicle, and expand the application scenario of target tracking.

[0073] Embodiment 3

[0074] Figure 3 is a flow chart of another tracking frame maintaining method provided according to Embodiment 3 of the present invention. The embodiment of the present invention is a specific embodiment of the above-mentioned invention embodiment, and introduces the comparison process of the first motion trajectory and the second motion trajectory. Figure 3 The method provided in the embodiment of the present invention specifically includes the following steps:

[0075] Step 310: Acquire the target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle.

[0076] Step 320: When the motion state is stationary, obtain a first motion trajectory of the target vehicle in the video data and a second motion trajectory of surrounding vehicles corresponding to the target vehicle.

[0077] Step 330: extract a first trajectory point and a second trajectory point corresponding to the target vehicle in the first motion trajectory, wherein the first trajectory point is earlier than the second trajectory point in recording time of the video data.

[0078] The first trajectory point and the second trajectory point may be trajectory points constituting the first motion trajectory, the first trajectory point and the second trajectory point may mark the location information of the target vehicle, and the first trajectory point may be recorded earlier than the second trajectory point.

[0079] In the embodiment of the present invention, the first motion trajectory of the target vehicle may be composed of two trajectory points, wherein the trajectory point recorded earlier may be recorded as the first trajectory point, and the other trajectory point may be recorded as the second trajectory point.

[0080] Step 340: determine the shortest distance between the first trajectory point and the second trajectory point and the head trajectory point in the second motion trajectory as the second shortest distance.

[0081] Among them, the head trajectory point can be a trajectory point located on the head in the second motion trajectory. It can be understood that the head trajectory point can be the most recently recorded trajectory point in the second motion trajectory. The head trajectory point can be the end position of the second motion trajectory. The second shortest distance can be the distance from the projection point of the trajectory point in the first motion trajectory on the second motion trajectory to the head trajectory point in the second motion trajectory. The second shortest distance can be the shortest distance from the first trajectory point or the second trajectory point to the head trajectory point in the second motion trajectory.

[0082] Specifically, the first trajectory point and the second trajectory point can be projected onto the second motion trajectory to obtain the projection point of the first trajectory point and the projection point of the second trajectory point. The shortest distance from the projection point of the first trajectory point to the head trajectory point in the second motion trajectory and the shortest distance from the projection point of the second trajectory point to the head trajectory point in the second motion trajectory can be determined respectively. The shortest distances corresponding to the first trajectory point and the second trajectory point can be used as the second shortest distance. It can be understood that there can be multiple second motion trajectories, and the second shortest distances from the first trajectory point and the second trajectory point to each corresponding second motion trajectory can be determined respectively.

[0083] Step 350: for the same second motion trajectory, compare the second shortest distances corresponding to the first trajectory points and the second trajectory points respectively to obtain a comparison result.

[0084] In an embodiment of the present invention, a second shortest distance generated based on the same second motion trajectory can be obtained, and the above-mentioned second shortest distance can include the second shortest distance corresponding to the first trajectory point and the second shortest distance corresponding to the second trajectory point. The second shortest distance of the first trajectory point can be compared with the second shortest distance of the second trajectory point to determine the distance relationship between the two. The comparison result of the second shortest distance of the first trajectory point and the second shortest distance of the second trajectory point can be used as the comparison result of the first motion trajectory and the second motion trajectory.

[0085] Step 360: If the comparison result is that the second shortest distance of the second trajectory point is greater than the second shortest distance of the first trajectory point, the surrounding vehicles corresponding to the second motion trajectory are determined as pending vehicles.

[0086] Specifically, when the comparison result of the first motion trajectory and the second motion trajectory is that the second shortest distance of the second trajectory point is greater than the second shortest distance of the first trajectory point, it can be determined that the surrounding vehicles corresponding to the second motion trajectory are located in front of the target vehicle, and the surrounding vehicles corresponding to the second motion trajectory can be marked as pending vehicles, which can be the vehicle used to select the target front vehicle.

[0087] Step 370: Determine the total distance from each pending vehicle to the target vehicle, and determine the pending vehicle with the smallest total distance as the target front vehicle.

[0088] Specifically, the total distance from the pending vehicle to the target vehicle can be the distance between the vehicle and the target vehicle. The total distance can be obtained through the pixel distance of the video data, or determined by comparing the first motion trajectory of the target vehicle with the second motion trajectory of the pending vehicle. The total distance from each pending vehicle to the target vehicle can be determined, and the total distance with the minimum value can be selected, and the pending vehicle corresponding to the total distance with the minimum value is used as the target preceding vehicle of the target vehicle.

[0089] Step 380: Determine that the motion state of the target vehicle in front is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

[0090] In an embodiment of the present invention, a target vehicle that has lost its tracking frame is determined in video data, the motion state of the target vehicle is obtained, and when the motion state of the target vehicle is determined to be stationary, a first motion trajectory of the target vehicle and a second motion trajectory of surrounding vehicles of the target vehicle are extracted from the video data, and the second shortest distances from the first trajectory point and the second trajectory point in the first motion trajectory to the head trajectory point in the second motion trajectory are determined, and the second shortest distances of the second motion trajectories are compared to obtain a comparison result. When the comparison result is that the second shortest distance of the second trajectory point is greater than the second shortest distance of the first trajectory point, the surrounding vehicles corresponding to the second motion trajectory are determined as pending vehicles, the total distance between the pending vehicles and the target vehicle is determined, and the pending vehicle with the smallest total distance is determined as the target front vehicle. When the motion state of the target front vehicle is stationary, it is determined The target vehicle is determined to be in a queue state, a historical tracking frame of the target vehicle in the video data is obtained, and the lost tracking frame is compensated for the target vehicle according to the historical tracking frame. The embodiment of the present invention determines the positional relationship between the surrounding vehicles and the target vehicle through a first track point of the first motion track and a distance comparison result from the second track point to the second motion track, thereby obtaining the target front vehicle located in front of the target vehicle. When it is determined that the target vehicle is in a queue state based on the motion states of the target front vehicle and the target vehicle, the lost tracking frame is compensated by extracting the historical tracking frame of the target vehicle from the video data, which can solve the problem of missing vehicle tracking frames in the video data. Compensating the tracking frame for the target vehicle by using the historical tracking frame can reduce the probability of loss of the tracked target, help improve the accuracy of subsequent target detection, and expand the application scenarios of target tracking.

[0091] Further, based on the above-mentioned embodiment of the invention, determining the total distance from each pending vehicle to the target vehicle includes:

[0092] For each pending vehicle, the shortest distances from the first trajectory point and the second trajectory point to the corresponding second motion trajectory are determined as the first shortest distance; for each pending vehicle, the sum of the first shortest distance of the first trajectory point, the first shortest distance of the second trajectory point and the second shortest distance of the first trajectory point corresponding to the pending vehicle is taken as the total distance of the pending vehicle.

[0093] The first shortest distance may be the shortest distance from the first trajectory point or the second trajectory point to the second motion trajectory, and the shortest distance may be the shortest straight-line distance from the first trajectory point or the second trajectory point to any position on the trajectory line within the second motion trajectory.

[0094] Specifically, for each pending vehicle, the shortest straight-line distances from the first trajectory point and the second trajectory point to the second motion trajectory of the pending vehicle can be determined as the first shortest distance, and the second shortest distance of the corresponding first trajectory point of each pending vehicle can be extracted. The sum of the first shortest distance of the first trajectory point of the pending vehicle, the second shortest distance of the second trajectory point, and the second shortest distance of the first trajectory point can be used as the total distance of the pending vehicle.

[0095] Embodiment 4

[0096] Figure 4 is a flow chart of another tracking frame maintenance method provided according to Embodiment 4 of the present invention. The embodiment of the present invention is a specific embodiment based on the above embodiment, and introduces the missing repair process of the tracking frame. Figure 4 The method provided in the embodiment of the present invention specifically includes the following steps:

[0097] Step 410: Acquire the target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle.

[0098] Step 420: When the motion state is stationary, obtain a first motion trajectory of the target vehicle in the video data and a second motion trajectory of surrounding vehicles corresponding to the target vehicle.

[0099] Step 430: determine the target preceding vehicle among the surrounding vehicles according to the distance comparison result between the first motion trajectory and the second motion trajectory.

[0100] Step 440: When the moving state of the target front vehicle is stationary, it is determined that the target vehicle is in a queue state, and a video frame previous to the current video frame in which the tracking frame is missing is obtained in the video data.

[0101] Among them, the current video frame can be a video frame in which the tracking frame is missing in the video data at the current moment, and the missing tracking frame in the current video frame can correspond to the target vehicle. The previous video frame can be a video frame in the video data that is one frame before the current video frame, and the previous video frame may not be missing the tracking frame of the target vehicle.

[0102] In an embodiment of the present invention, when it is determined that the motion state of the target front vehicle is stationary, the target vehicle is in a queue state, and its previous video frame can be obtained as the previous video frame in the video data based on the current video frame at the current moment.

[0103] Step 450: extract the historical tracking frame of the target vehicle in the previous video frame.

[0104] Specifically, a historical tracking frame corresponding to the target vehicle may be extracted from a previous video frame, and the historical tracking frame may include at least tracking frame attribute information such as position attributes, size attributes, and category attributes.

[0105] Step 460: Add a tracking frame for the target vehicle in the current video frame according to the tracking frame attribute information of the historical tracking frame.

[0106] Among them, the tracking frame attribute information can be information for target tracking algorithm processing of video data. The tracking frame attribute information can identify the attributes of each tracking frame in the video data. The tracking frame attribute information may include position attributes, size attributes, and category attributes of the tracking frame. The position attribute can identify the position of the tracking frame in the video frame, the size attribute can identify the size of the tracking frame, and the category attribute can identify the category of the image content framed by the tracking frame, such as the type of vehicle, or specific information of the vehicle.

[0107] In the embodiment of the present invention, the tracking frame attribute information of the historical tracking frame can be extracted in the previous video frame, and the tracking frame can be generated in the current video frame according to the tracking frame attribute information, thereby maintaining the lost tracking frame.

[0108] The technical solution of the embodiment of the present invention determines the target vehicle with a lost tracking frame in the video data, obtains the motion state of the target vehicle, and when it is determined that the motion state of the target vehicle is stationary, determines the first motion trajectory of the target vehicle and the second motion trajectory of the surrounding vehicles of the target vehicle in the video data, obtains the distance comparison result by comparing the first motion trajectory and the second motion trajectory, and determines the target front vehicle of the target vehicle according to the distance comparison result, and when the motion state of the target front vehicle is stationary, determines that the target vehicle is in a queue state, obtains the previous video frame of the current video frame with a missing tracking frame in the video data, extracts the historical tracking frame of the target vehicle in the previous video frame, generates the lost tracking frame of the target vehicle in the current video frame through the tracking frame attribute information of the historical tracking frame, thereby compensating for the data of the lost tracking frame in the target tracking process, which can reduce the probability of loss of the tracked target, help improve the accuracy of subsequent target detection, and expand the application scenarios of target tracking.

[0109] Embodiment 5

[0110] Currently, in the field of smart transportation, pure video multi-target tracking adopts the Tracking-by-detection algorithm. The tracking effect depends largely on the detection results of the video detector on the vehicle target. See Figure 5 When the vehicle queue is too dense, the target vehicle may not be accurately detected due to the obstruction of the vehicle in front. This missed detection phenomenon will cause the motor vehicle target to lose tracking, thus affecting the subsequent video analysis. In this case, the dense queue of motor vehicles makes it difficult to maintain the target tracking state, and the queue data is inaccurate. As a result, the motor vehicle target tracking ID will be frequently switched, affecting the analysis of subsequent traffic statistics, vehicle capture and other data. The processing flow based on the Tracking-by-detection idea includes the following processes:

[0111] 1. General processing flow of non-queuing state

[0112] (1) Detection module. Traditional target detection relies on image processing and computer vision. In recent years, deep learning has been widely used in the field of target detection. Its core lies in extracting effective features from images through models such as CNN, and performing target classification and position regression at the same time. Representative deep learning detection models include Faster R-CNN, YOLO, SSD, etc., which usually have higher detection accuracy. The general workflow of the detection module is: first read each frame of video, then input it into the deep learning target detector for processing, and obtain the tracking boxes of multiple motor vehicle targets in the current video frame.

[0113] (2) Tracking module. Each motor vehicle target will have a corresponding tracker. The IOU intersection-over-union ratio and Hungarian algorithm are used to optimally match the acquired target tracking frames with each previous tracker. If the match is successful, the Kalman filter is used in combination with the latest tracking frame coordinates to update the position status of the target vehicle. If the tracking frame is not matched successfully, a new tracker is assigned to the tracking frame, indicating that it is a new motor vehicle target.

[0114] 2. Processing flow of queue status

[0115] If the motor vehicle target in the video data does not match the tracking frame, it may be missed due to dense queue occlusion, and the queue status is judged. Assuming that the motor vehicle target that is lost is A, the method for judging the queue status of target A is as follows:

[0116] 1. First, make sure that target A must be stationary.

[0117] 2. Find the vehicle in front of target A. After being processed by the detection module and the tracking module, all motor vehicle targets have a track set saved, which is a set of midpoints of the bottom edges of the tracking box that the motor vehicle has passed.

[0118] 3. Target A takes the latest two trajectory points P_2 and P_1, P_2 is the farther trajectory point, and P_1 is the latest trajectory point.

[0119] 4. Traverse all other motor vehicle targets in the current frame, record them as X, and take the latest 8 trajectory points of X target from back to front {(x 8 ,y 8 ),(x 7 ,y 7 )......(x 2 ,y 2 ),(x 1 ,y 1 )}. These trajectory points form non-smooth trajectory segments (7 segments in total), (x 8 ,y 8 ) is the farthest trajectory point, (x 1 ,y 1 ) is the latest track point.

[0120] 5. Calculate two distances between the two trajectory points P_2 and P_1 of target A and the trajectory line segment of target X: the first shortest distance Dist_1 and the second shortest distance Dist_2. The first shortest distance Dist_1 is the distance from the trajectory point of target A to the trajectory line segment of target X, which can have three cases, such as Figure 6 The second shortest distance is from the projection point P of the target trajectory point A on the target trajectory segment X to the latest trajectory point (x 1 ,y 1 ) can be Figure 7 shown.

[0121] 6. Determine whether target A is behind target X. If target A is behind target X, then Dist_2 of P_2 should be greater than Dist_2 of P_1, otherwise the determination of target X ends.

[0122] 7. If the X target is in front of the A target, add the Dist_1 distances of P_2 and P_1, and add the Dist_2 distance of P_1 to get the total distance Dist between the A target and the X target.

[0123] 8. The X target with the smallest total distance Dist is the vehicle directly in front of the A target, and this vehicle is recorded as A-Front.

[0124] 9. If the A-Front target is in a stationary state, it is determined that the A target is currently in a queue state, otherwise the A target is in a non-queue state.

[0125] 10. Through the above round of judgment, the stationary vehicles behind target A can be obtained at the same time. If you need to find the vehicle in front of target A again later, these stationary vehicles behind it can be filtered out.

[0126] Through the above steps, it can be determined whether the target A that has lost detection is in the queue. If the target A is in the queue, the current tracking frame position of the target A can be input as the tracking frame into the tracker of the target A to continue to update the position information. The complete schematic diagram of multiple vehicles is shown as follows: Figure 8 shown.

[0127] Further, based on the above-mentioned embodiments of the invention, the determination of whether the motor vehicle target is in a moving state or a stationary state may be performed as follows:

[0128] (1) Normalized dimensionless pixel speed. Since each motor vehicle target has a tracker, the pixel speed of the motor vehicle target in the X and Y directions can be obtained by using Kalman filtering for the tracker. Since two-dimensional images are larger when near and smaller when far, the speed of motor vehicle targets far away in the video will be smaller when calculated by Kalman filtering. In order to eliminate this effect, the speed is divided by the width of the target tracking box to obtain the dimensionless normalized pixel speed. When the pixel speed is lower than the set speed threshold, it is determined that the motor vehicle target is stationary.

[0129] (2) Detect moving targets using background frame difference. Using the frame difference method for background detection can obtain the foreground in motion in the video. When the moving motor vehicle target gradually slows down to a standstill, the background frame difference motion detector will detect that the foreground in the motor vehicle target tracking area gradually disappears. When the foreground disappears completely, it is determined that the target is stationary.

[0130] For details, see Fig. 9 , can effectively maintain the tracking frame of multiple target locomotives in the queue state, which can make up for Figure 5 The lost tracking frame of the motor vehicle target can be calculated in the scenario of densely queued motor vehicles to determine whether the current vehicle is in a queue, so as to repair the lost tracking frame.

[0131] Embodiment 6

[0132] Fig.10 1 is a schematic diagram of a tracking frame maintaining device according to Embodiment 6 of the present invention. Fig.10 As shown, the device comprises:

[0133] The missing detection module 510 is used to obtain the target vehicle whose tracking frame is missing according to the video data and determine the motion state of the target vehicle.

[0134] The maintenance trigger module 520 is used to obtain a first motion track of the target vehicle and a second motion track of surrounding vehicles corresponding to the target vehicle in the video data when the motion state is static.

[0135] The preceding vehicle determining module 530 is configured to determine a target preceding vehicle among surrounding vehicles according to a distance comparison result between the first motion trajectory and the second motion trajectory.

[0136] The tracking frame adding module 540 is used to determine that the motion state of the target front vehicle is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

[0137] In an embodiment of the present invention, a target vehicle that has lost a tracking frame is determined in video data by a missing detection module, and the motion state of the target vehicle is obtained. When a maintenance trigger module determines that the motion state of the target vehicle is stationary, a first motion trajectory of the target vehicle and a second motion trajectory of vehicles surrounding the target vehicle are determined in the video data. A leading vehicle determination module obtains a distance comparison result by comparing the first motion trajectory and the second motion trajectory, and determines a target leading vehicle of the target vehicle according to the distance comparison result. When the motion state of the target leading vehicle is stationary, a tracking frame adding module obtains a historical tracking frame of the target vehicle in the video data, thereby compensating for the data of the lost tracking frame during the target tracking process, which can reduce the probability of loss of the tracked target and help maintain the target tracking state.

[0138] In some embodiments of the invention, the missing detection module 510 determines the motion state of the target vehicle, including at least one of the following:

[0139] Determining the motion state as stationary based on the pixel velocity of the target vehicle;

[0140] The motion state is determined to be stationary based on the background frame difference of the target vehicle.

[0141] In some embodiments of the invention, the missing detection module 510 determines the motion state as stationary based on the pixel speed of the target vehicle, including:

[0142] At least two frames of vehicle images of a target vehicle are determined in video data; a vertical pixel speed and a horizontal pixel speed of the target vehicle are determined according to the vehicle images; the vertical pixel speed and the horizontal pixel speed are normalized to obtain a pixel speed; and when the pixel speed is less than a threshold speed, the motion state is determined to be stationary.

[0143] In some inventive embodiments, the maintenance trigger module 520 includes:

[0144] The first trajectory unit is used to extract at least a first number of historical video frames closest to the current moment in the video data, and form a first motion trajectory according to the vehicle position of the target vehicle in each historical video frame.

[0145] The second trajectory unit is used to extract the trajectory set of each surrounding vehicle in the video data, and select trajectory points that meet at least a second number in the trajectory set to form a second motion trajectory, wherein the distance between two adjacent trajectory points is at least greater than a threshold distance.

[0146] In some inventive embodiments, the preceding vehicle determination module 530 includes:

[0147] The trajectory extraction unit is used to extract a first trajectory point and a second trajectory point corresponding to the target vehicle in the first motion trajectory, wherein the first trajectory point is earlier than the second trajectory point in recording time of the video data.

[0148] The distance determination unit is used to determine the shortest distance between the first trajectory point and the second trajectory point and the head trajectory point in the second motion trajectory as the second shortest distance.

[0149] The position comparison unit is used to compare the second shortest distances corresponding to the first trajectory point and the second trajectory point respectively for the same second motion trajectory to obtain a comparison result.

[0150] The pending selection unit is used to determine the surrounding vehicles corresponding to the second motion trajectory as the pending vehicles if the comparison result is that the second shortest distance of the second trajectory point is greater than the second shortest distance of the first trajectory point.

[0151] The preceding vehicle selection unit is used to determine the total distance from each pending vehicle to the target vehicle, and determine the pending vehicle with the smallest total distance as the target preceding vehicle.

[0152] In some embodiments of the invention, determining the total distance from each pending vehicle to the target vehicle in the preceding vehicle selection unit includes:

[0153] For each undetermined vehicle, determine the shortest distance from the first trajectory point and the second trajectory point to the corresponding second motion trajectory as the first shortest distance;

[0154] For each pending vehicle, the sum of the first shortest distance of the first trajectory point, the first shortest distance of the second trajectory point and the second shortest distance of the first trajectory point corresponding to the pending vehicle is taken as the total distance of the pending vehicle.

[0155] In some inventive embodiments, the tracking frame adding module 540 includes:

[0156] The video frame extraction unit is used to determine that the target vehicle is in a queue state when the moving state of the target front vehicle is stationary, and obtain a previous video frame of the current video frame with a missing tracking frame in the video data.

[0157] The tracking frame extraction unit is used to extract the historical tracking frame of the target vehicle in the previous video frame.

[0158] The patch adding unit is used to add a tracking frame for the target vehicle in the current video frame according to the tracking frame attribute information of the historical tracking frame.

[0159] The tracking frame maintaining device provided in the embodiment of the present invention can execute the tracking frame maintaining method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0160] Embodiment 7

[0161] Fig.11 : is a schematic diagram of the structure of an electronic device that implements the tracking frame maintenance method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0162] like Fig.11 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0163] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0164] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a tracking frame maintenance method.

[0165] In some embodiments, the tracking frame maintenance method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the tracking frame maintenance method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the tracking frame maintenance method in any other appropriate manner (e.g., by means of firmware).

[0166] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0168] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0171] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0172] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0173] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A tracking frame maintaining method, characterized in that: The method comprises: Acquire a target vehicle whose tracking frame is missing according to the video data, and determine the motion state of the target vehicle; When the motion state is stationary, obtaining a first motion trajectory of the target vehicle and a second motion trajectory of surrounding vehicles corresponding to the target vehicle in the video data; Determining a target preceding vehicle among the surrounding vehicles according to a distance comparison result between the first motion trajectory and the second motion trajectory; It is determined that the motion state of the target front vehicle is stationary, and the target vehicle is in a queue state, and the historical tracking frame of the target vehicle in the video data is used as the missing tracking frame of the target vehicle.

2. The method according to claim 1, characterized in that: Determining the motion state of the target vehicle includes at least one of the following: Determining the motion state as stationary based on the pixel speed of the target vehicle; The motion state is determined to be stationary based on the background frame difference of the target vehicle.

3. The method according to claim 2, characterized in that: The determining that the motion state is stationary based on the pixel speed of the target vehicle includes: Determining at least two frames of vehicle images of the target vehicle in the video data; Determine a vertical pixel speed and a horizontal pixel speed of the target vehicle according to the vehicle image; Normalizing the vertical pixel speed and the horizontal pixel speed to obtain a pixel speed; The motion state is determined to be stationary when the pixel speed is less than a threshold speed.

4. The method according to claim 1, characterized in that: The obtaining of a first motion track of the target vehicle in the video data and a second motion track of surrounding vehicles corresponding to the target vehicle includes: Extracting at least a first number of historical video frames closest to the current moment from the video data, and forming the first motion trajectory according to the vehicle position of the target vehicle in each of the historical video frames; A trajectory set of each of the surrounding vehicles is extracted from the video data, and trajectory points satisfying at least a second number are selected from the trajectory set to form the second motion trajectory, wherein a distance between two adjacent trajectory points is at least greater than a threshold distance.

5. The method according to claim 1, characterized in that: The determining of the target preceding vehicle among the surrounding vehicles according to the comparison result between the first motion trajectory and the second motion trajectory includes: Extracting a first trajectory point and a second trajectory point corresponding to the target vehicle in the first motion trajectory, wherein the first trajectory point is earlier than the second trajectory point in recording time of the video data; Determine the shortest distance between the first trajectory point and the second trajectory point and the head trajectory point in the second motion trajectory as the second shortest distance; For the same second motion trajectory, comparing the first trajectory point and the second trajectory point respectively corresponding to the second shortest distance to obtain the comparison result; If the comparison result is that the second shortest distance of the second trajectory point is greater than the second shortest distance of the first trajectory point, the surrounding vehicle corresponding to the second motion trajectory is determined as a pending vehicle; The total distance from each of the pending vehicles to the target vehicle is determined, and the pending vehicle with the smallest total distance is determined as the target preceding vehicle.

6. The method according to claim 5, characterized in that: The determining of the total distance from each of the pending vehicles to the target vehicle comprises: For each of the pending vehicles, determining the shortest distances from the first trajectory point and the second trajectory point to the corresponding second motion trajectory as the first shortest distance; For each of the pending vehicles, the sum of the first shortest distance of the first trajectory point, the first shortest distance of the second trajectory point, and the second shortest distance of the first trajectory point corresponding to the pending vehicle is taken as the total distance of the pending vehicle.

7. The method according to claim 1, characterized in that: The determining that the motion state of the target front vehicle is stationary, the target vehicle is in a queue state, and using the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle includes: When the moving state of the target front vehicle is stationary, determining that the target vehicle is in a queue state, and obtaining a video frame before the current video frame in which the tracking frame is missing in the video data; Extracting the historical tracking frame of the target vehicle in the previous video frame; The tracking frame is added to the target vehicle in the current video frame according to the tracking frame attribute information of the historical tracking frame.

8. A tracking frame maintaining device, characterized in that: The device comprises: A missing detection module is used to obtain a target vehicle whose tracking frame is missing according to video data, and determine the motion state of the target vehicle; A maintenance trigger module is used to obtain a first motion track of the target vehicle and a second motion track of surrounding vehicles corresponding to the target vehicle in the video data when the motion state is stationary; A preceding vehicle determining module, configured to determine a target preceding vehicle among the surrounding vehicles according to a distance comparison result between the first motion trajectory and the second motion trajectory; The tracking frame adding module is used to determine that the motion state of the target front vehicle is stationary, and the target vehicle is in a queue state, and use the historical tracking frame of the target vehicle in the video data as the missing tracking frame of the target vehicle.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the tracking frame maintaining method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tracking frame maintaining method according to any one of claims 1 to 7 when the processor executes the computer instructions.

Images