Target tracking method, device, equipment and storage medium

Abstract

This specification discloses a target tracking method, apparatus, device, and storage medium. The method obtains a target object by performing target detection only on the first video frame of a video to be processed, and then determines the position of the target object in each second video frame based on the position of the target object in the first video frame and the obtained motion vector corresponding to each video frame in the video to be processed, thereby tracking the target object in each video frame of the video to be processed. It can be seen that by performing high-precision target detection only on the first video frame of the video to be processed, the accuracy of target detection is guaranteed under limited computing power conditions. At the same time, the position of the target object in the second video frame other than the first video frame is determined using the motion vector to achieve tracking of the target object, while taking into account the real-time performance of target tracking.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a target tracking method, device, equipment, and storage medium. Background Art

[0002] With the development of visual computing technology, target tracking technology has been widely used in video broadcasting, security monitoring, unmanned driving and other fields. The main task of target tracking is to continuously, in real time, and accurately locate the target object to be tracked in a video sequence through intelligent devices. Among them, target tracking can adopt a dense tracking method, that is, using a target detection model to perform target detection on each frame of the video, determine the position of the tracked target object in each frame, and thus achieve target tracking. However, due to the limited computing power of intelligent devices (such as smart cameras), the frame rate of the target detection algorithm used to calculate the video image is much lower than the frame rate of the video. Therefore, it is impossible to achieve target tracking by performing target detection calculations on each frame of the video. This reduces the real-time performance of target tracking and may even make it impossible to track the target object.

[0003] In the existing technology, in order to achieve target tracking on smart devices under limited computing power conditions, a streamlined target detection model is usually adopted to reduce the calculation time of the target detection model and increase the frame rate of the video image calculation using the target detection algorithm, thereby improving the real-time performance of target tracking.

[0004] However, streamlining the target detection model will lead to a decrease in the accuracy of the target detection model, thereby reducing the accuracy of target tracking. Summary of the Invention

[0005] This specification provides a target tracking method, device, equipment and storage medium to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a target tracking method, including:

[0008] Obtaining a video to be processed, and determining a first video frame and a second video frame in the video to be processed;

[0009] Performing target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame; and determining a motion vector corresponding to each video frame in the video to be processed;

[0010] determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed;

[0011] The target object is tracked according to the position of the target object in the first video frame and the position of the target object in the second video frame.

[0012] Optionally, determining the first video frame and the second video frame in the video to be processed specifically includes:

[0013] Determining a frame extraction frequency based on the frame rate of the video to be processed and the frame rate of the pre-trained object detection model;

[0014] Frame extraction is performed on the video to be processed according to the frequency, the extracted video frames are used as first video frames, and video frames other than the first video frames are used as second video frames.

[0015] Optionally, determining the motion vector corresponding to each video frame in the video to be processed specifically includes:

[0016] Performing multi-scale adjustment on the video to be processed to obtain multiple videos to be processed at different scales;

[0017] For each scale of the video to be processed, each video frame in the video to be processed at that scale is segmented to obtain pixel blocks of each video frame at that scale; and the motion vector corresponding to each pixel block in each video frame at that scale is obtained as the motion vector corresponding to the video frame at that scale; the motion vector corresponding to a pixel block in a video frame is used to represent the displacement of the pixel block from the video frame to the next video frame.

[0018] Optionally, the target object in the first video frame includes a plurality of target objects having a superior-subordinate relationship; wherein the pixel block covered by the upper-level target object in the video frame includes the pixel block covered by the lower-level target object in the video frame;

[0019] For each level of target object, determine a designated scale corresponding to the target object at that level from among the multiple different scales, wherein a motion vector corresponding to a pixel block covered by the target object at that level exists in the video to be processed at the designated scale, and the designated scale is the smallest among the multiple different scales; and the motion vector corresponding to the pixel block covered by the target object at that level in the video to be processed at the designated scale is: the designated motion vector corresponding to the target object at that level;

[0020] Determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed specifically includes:

[0021] Selecting a current scale in order from small to large according to the different scales, and determining a target object in the first video frame with the selected current scale as a specified scale as the current target object;

[0022] Determining a position of the current target object in a second video frame at the current scale according to the position of the current target object in the first video frame at the current scale and a designated motion vector corresponding to the current target object;

[0023] If there is an unselected scale, the current scale is selected in order from the smallest to the largest scale, and the position of the target object in the first video frame with the selected current scale as the specified scale is determined in the second video frame until there is no unselected scale.

[0024] Optionally, determining the position of the current target object in the second video frame at the current scale according to the position of the current target object in the first video frame at the current scale and a specified motion vector corresponding to the current target object specifically includes:

[0025] If the current target object does not have a previous-level target object, determining a position of the current target object in a second video frame at the current scale according to a position of the current target object in a first video frame at the current scale and a designated motion vector corresponding to the current target object;

[0026] If the current target object has a parent target object, mapping the position of the parent target object of the current target object in the corresponding second video frame of the specified scale to the second video frame of the current scale corresponding to the current target object to determine the position of the parent target object of the current target object in the second video frame of the current scale;

[0027] Determining a relative motion vector corresponding to the current target object according to a designated motion vector corresponding to the current target object and a designated motion vector corresponding to a target object one level above the current target object;

[0028] The position of the current target object in the second video frame at the current scale is determined according to the position of the previous level target object of the current target object in the second video frame at the current scale and the relative motion vector corresponding to the current target object.

[0029] Optionally, the method further includes:

[0030] When there is no unselected scale, among the different scales, restoring the second video frame with the largest scale to a second video frame with an original scale; wherein the original scale includes the acquired scale of the video to be processed;

[0031] Positions of target objects at all levels in the first video frame in the second video frame with the largest scale are determined according to the positions of target objects at all levels in the first video frame in the second video frame with the original scale.

[0032] Optionally, determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed specifically includes:

[0033] If the second video frame is adjacent to the first video frame, determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed;

[0034] If the second video frame is not adjacent to the first video frame, the method further includes:

[0035] Determine the position of the target object in the second video frame based on the position of the target object in the video frame previous to the second video frame and the motion vector corresponding to the second video frame; wherein the position of the target object in the video frame previous to the second video frame is determined based on the position of the target object in the first video frame and the motion vector corresponding to the video frame previous to the second video frame.

[0036] This specification provides a target tracking device, including:

[0037] A video frame determination module is used to obtain a video to be processed and determine a first video frame and a second video frame in the video to be processed;

[0038] a target object determination module, configured to perform target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame;

[0039] A motion vector determination module, configured to determine a motion vector corresponding to each video frame in the video to be processed;

[0040] a position determination module, configured to determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed;

[0041] A tracking module is used to track the target object according to the position of the target object in the first video frame and the position of the target object in the second video frame.

[0042] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the target tracking method described above is implemented.

[0043] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the target tracking method when executing the program.

[0044] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0045] In the target tracking method provided in this specification, target detection is performed only on the first video frame of the video to be processed to obtain the target object. Based on the position of the target object in the first video frame and the motion vector corresponding to each video frame in the video to be processed, the position of the target object in each second video frame is determined, thereby tracking the target object in each video frame of the video to be processed. It can be seen that by performing high-precision target detection only on the first video frame of the video to be processed, the accuracy of target detection is guaranteed under limited computing power conditions. At the same time, the position of the target object in the second video frame other than the first video frame is determined using the motion vector to achieve tracking of the target object, taking into account the real-time performance of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0047] Figure 1 A flowchart of a target tracking method in this specification;

[0048] Figure 2A A schematic diagram of the relationship between various target objects in this specification;

[0049] Figure 2B A schematic diagram of the relationship between various target objects in this specification;

[0050] Figure 2C This is a schematic diagram of target objects at various levels with multiple scales in this specification;

[0051] Figure 3 A flowchart of a target tracking method in this specification;

[0052] Figure 4 is a schematic diagram of a first video frame and a second video frame at multiple scales in this specification;

[0053] Figure 5 A flowchart of a target tracking method in this specification;

[0054] Figure 6A This is a schematic diagram of determining a position of a target object in a second video frame in this specification;

[0055] Figure 6B This is a schematic diagram of determining a position of a target object in a second video frame in this specification;

[0056] Figure 7 A flowchart of a target tracking method in this specification;

[0057] Figure 8 is a schematic diagram of a video frame sequence in a video to be processed in this specification;

[0058] Figure 9 A schematic diagram of a target tracking device provided in this specification;

[0059] Figure 10 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0061] Target tracking refers to the process of analyzing and processing video captured by smart devices, leveraging the captured video information to stably track a target. Once a target is identified, its motion parameters, such as position, velocity, and acceleration, can be determined, ultimately yielding its characteristic parameters. In the military, target tracking technology is widely used in areas such as precision missile guidance, autonomous robot navigation, drone landing, and electro-optical tracking at target ranges. In civilian applications, this technology also has important applications in intelligent video surveillance, intelligent traffic control, and medical imaging diagnostics.

[0062] Target tracking can be viewed as a series of target detections. For example, in a surveillance video, the target is a pedestrian. Each frame of the video is extracted and target detection is performed on each frame. By detecting the target pedestrian's position in each frame, the target pedestrian's movement process in the surveillance video can be determined, thereby achieving tracking of the target pedestrian. Furthermore, the target pedestrian's displacement is calculated by the change in coordinates between two frames, and the target pedestrian's movement speed is obtained by combining the time interval between the two frames. This method of performing target detection on each frame to achieve target tracking is computationally intensive because it requires processing every frame of the video. Furthermore, the frame rate of a high-precision target detection algorithm is far lower than the frame rate of the video. In other words, using a high-precision target detection algorithm for target tracking cannot track the target in the video in real time. However, while using a streamlined target detection model can reduce the target detection model's computation time and increase the frame rate of the target detection algorithm's computation on the video image, it also results in a decrease in the accuracy of the target detection model, which in turn reduces the accuracy of target tracking.

[0063] To achieve both accurate and real-time target tracking, this method leverages the video encoding capabilities typically found on smart devices to obtain motion vectors for each frame of raw, unprocessed video. Object detection is performed on only a small portion of the video frames, and the motion vectors are used to predict the target's displacement between detections, enabling tracking of the target in the video.

[0064] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0065] Figure 1 The following is a flow chart of a target tracking method in this specification, which specifically includes the following steps:

[0066] S100: Obtain a video to be processed, and determine a first video frame and a second video frame in the video to be processed.

[0067] The processed video is generated after image information is captured by a smart device. Depending on the application, the processed video can be stored and played locally on the smart device or transferred to other devices for storage and playback. For applications where the processed video needs to be transmitted to other devices, because the transmission network cannot accommodate the large data volume of the processed video, a video encoding module must be configured in the smart device to encode and compress the original video before transmission. Smart devices include smart cameras, smartphones, and other devices capable of capturing images and generating videos.

[0068] In the embodiments of this specification, the device executing the target tracking method can be an intelligent device that generates video or another device that receives video (such as a server), and this specification does not limit this. However, it should be noted that since the computing power of the intelligent device that generates video to achieve target tracking is more limited than that of other devices that receive video, this method is more suitable for achieving high-precision and high-real-time target tracking on intelligent devices with limited computing power.

[0069] Furthermore, in this step, a portion of the video frames in the video to be processed is extracted as the video frames for subsequent target detection. These extracted video frames are used as the first video frames, and the video frames in the video to be processed other than the first video frames are used as the second video frames. Typically, to improve the real-time performance of target tracking, the number of first video frames can be smaller than the number of second video frames. This specification does not limit the frequency of extraction of the first video frames.

[0070] S102: Perform target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame; and determine a motion vector corresponding to each video frame in the video to be processed.

[0071] The task of target detection is to obtain all target objects in the video frame and determine the category and position of the target objects. The target detection model used in the embodiments of this specification can be any existing target detection model, wherein the target detection model can be a machine learning model, which is not limited in this specification. In addition, it should be noted that the first video frame is used to determine the target object and the position of the target object in the first video frame through the target detection model. The second video frame is used to determine the position of the target object in the second video frame based on the position of the target object in the first video frame and the motion vector corresponding to the target object.

[0072] In addition, since the amount of data in the video to be processed is very large, once the transmission and storage of the video is involved, the original video needs to be encoded and compressed before transmission and storage. Normally, the smart device that captures images and generates videos can be configured with a video encoding module to encode the generated original video. This method uses the video encoding function that is usually configured on smart devices to obtain the motion vector corresponding to each video frame in the video to be processed. Target detection is performed on only a small number of video frames, and the motion vector is used to predict the position of the target object in other video frames that have not undergone target detection, thereby determining the position of the target object in each video frame of the video to be processed and achieving tracking of the target object in the video to be processed.

[0073] S104: Determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed.

[0074] In order to track the target object in the video to be processed, it is necessary to obtain the position of the target object in each video frame in the video to be processed. Since the target detection has been performed on the first video frame in the aforementioned step S102, it is also necessary to determine the position of the target object in the second video frame, that is, to determine the position of the target object in each video frame in the video to be processed.

[0075] Since the motion vector corresponding to the video frame can represent the displacement of the target object in the video frame from the current video frame to the next video frame, the position of the target object in the first video frame and the motion vector can be used to predict the position of the target object in the next video frame of the first video frame.

[0076] S106: Track the target object according to the position of the target object in the first video frame and the position of the target object in the second video frame.

[0077] According to the arrangement order of the video frames in the video to be processed, starting from the first video frame where the target has been detected, the motion vector corresponding to the video frame is used to determine the position of the target object in each second video frame in turn, thereby obtaining the position of the target object in each video frame in the video to be processed and realizing tracking of the target object.

[0078] In the target tracking method provided in this specification, target detection is performed only on the first video frame of the video to be processed to obtain the target object. Based on the position of the target object in the first video frame and the motion vectors corresponding to each video frame in the video to be processed, the position of the target object in each second video frame is determined, thereby tracking the target object in each video frame of the video to be processed. As can be seen, by performing target detection only on the first video frame of the video to be processed, the accuracy of target detection is guaranteed under limited computing power conditions. At the same time, the position of the target object in the second video frame other than the first video frame is determined using the motion vector to achieve tracking of the target object, taking into account the real-time performance of target tracking.

[0079] In the embodiments of this specification, Figure 1 The determination of the first video frame and the second video frame in the video to be processed shown in step S100 can be specifically implemented through the following steps.

[0080] First, obtain the frame rate of the pre-trained object detection model and the frame rate of the video to be processed.

[0081] Because object detection isn't performed on every frame of the video being processed in this embodiment, the object detection model used can be a more accurate model among existing object detection models. Typically, the frame rate of a more accurate object detection model is much lower than the frame rate of the video being processed. The frame extraction frequency is determined based on the frame rate of the video being processed and the frame rate of the pre-trained object detection model.

[0082] Then, the frequency of frame extraction is determined according to the frame rate of the video to be processed and the frame rate of the pre-trained target detection model.

[0083] Finally, the video to be processed is extracted according to the frequency, the extracted video frames are used as first video frames, and the video frames other than the first video frames are used as second video frames.

[0084] In practical applications, in order to further save computing power of smart devices, the frame extraction frequency can be further reduced while ensuring target tracking accuracy. The frame extraction frequency can be determined according to specific application scenarios and is not limited in this specification.

[0085] For example, the frame rate of the video to be processed is 50fps, which means that the interval between two video frames in the video to be processed is 20ms. The frame rate of the pre-trained target detection model is 10fps, which means that the running time of the target detection model for one video frame is 100ms. If the target detection model in the smart device is running at full capacity, one frame can be extracted every five frames as the first video frame for target detection. In order to save the computing power of the smart device, that is, if the target detection model does not need to run at full capacity, one frame can be extracted every six frames as the first video frame for target detection. At this time, since target detection with a running time of 100ms is performed only once every 120ms, the target detection model has 20ms of idle time every 120ms, saving 16% of computing power.

[0086] In the embodiments of this specification, Figure 1 The step S102 of determining the motion vector corresponding to each video frame in the video to be processed is specifically implemented through the following steps.

[0087] First, the video to be processed is subjected to multi-scale adjustment to obtain multiple videos to be processed of different scales.

[0088] Specifically, in order to avoid the situation in which the displacement of a target object with fewer covered pixel blocks between two video frames cannot be represented in a small-scale video to be processed, the video to be processed is subjected to multi-scale adjustment to obtain multiple videos of different scales to be processed, so as to subsequently determine the motion vectors corresponding to the video frames of multiple different scales, thereby being able to fully represent the displacement information of target objects at different levels.

[0089] Then, for each scale of the to-be-processed video, each video frame in the to-be-processed video of the scale is segmented to obtain pixel blocks of each video frame of the scale.

[0090] Finally, the motion vector corresponding to each pixel block in each video frame of the scale is obtained as the motion vector corresponding to the video frame of the scale; the motion vector corresponding to a pixel block in a video frame is used to represent the displacement of the pixel block from the video frame to the next video frame.

[0091] In actual applications, due to the large amount of video data, a video encoding module is usually configured in the smart device to encode the acquired video to reduce the amount of video data. The encoded video is then transmitted to the server or user end via a wired or wireless network and then decoded. This can reduce the network bandwidth required during the transmission process and reduce the storage space of the video.

[0092] Typically, the target object in two consecutive video frames does not change suddenly. Therefore, the position of the target object in the previous and / or next video frames in the time domain can be used to predict the position of the target object in the current video frame. In the inter-frame prediction process of video coding, the temporal correlation between video frames can be used to obtain the motion vector of the current video frame to achieve video frame compression.

[0093] Specifically, each video frame in the video being processed is divided into several non-overlapping pixel blocks, and the displacement direction and magnitude of all pixels within a pixel block are assumed to be the same. For each pixel block in the current video frame, the pixel block most similar to the pixel block is determined within a specified search range of the previous and / or next video frames of the current video frame according to a preset matching rule. This is referred to as the pixel block's matching block. The motion vector for the pixel block is then determined based on the displacement between the pixel block and its corresponding matching block.

[0094] In the embodiments of this specification, Figure 1 When performing target detection on the first video frame using the pre-trained target detection model shown in step S102, target detection may be performed on first video frames of multiple different scales, or may be performed on first video frames of only one scale among the multiple different scales.

[0095] For object detection in multiple first video frames of different scales: Using a pre-trained object detection model, target detection can be performed on multiple first video frames of different scales to obtain the target object's position in the first video frames of different scales. Using the target object's position in the first video frames of different scales and its corresponding motion vector, the target object's position in the second video frames of different scales can be determined.

[0096] For the case of performing target detection on a first video frame of one scale among multiple different scales: a first video frame of a scale that can be detected by a pre-trained target detection model can be selected from the first video frames of multiple different scales, and the pre-trained target detection model can be used to perform target detection on the first video frame of the selected scale to determine the position of the target object in the first video frame of the selected scale, so that the position of the target object in the second video frames of multiple different scales can be determined based on the positions of the target objects of multiple different levels in the first video frame of the selected scale, combined with the motion vectors corresponding to the video frames of multiple different scales and the relative motion vectors corresponding to the target objects of different levels. Then, among the multiple different scales, the second video frame with the largest scale is restored to the second video frame of the original scale, and finally the positions of the target objects of each level in the second video frame of the original scale are determined. Among them, the one scale selected from the multiple different scales can be any scale that can be detected by the pre-trained target detection model, and this specification does not limit this.

[0097] In the embodiments of this specification, Figure 1 When determining the position of the target object in the second video frame, as shown in step S104, it is necessary to determine the relationship between the target objects detected in the first video frame. The target objects can be independent of each other or have a hierarchical relationship. Therefore, the pre-trained target detection model used in the embodiments of this specification can directly detect all target objects when performing target detection on the first video frame.

[0098] Generally, for multiple target objects with independent relationships, the pixel blocks covered by each target object in the video frame are independent and do not intersect with each other, such as Figure 2A As shown in FIG1 , the target objects A1 and A2 are shown in FIG1 . It can be seen that the pixel blocks covered by the target object A1 do not intersect with the pixel blocks covered by the target object A2. At this time, the target objects A1 and A2 are independent of each other. For multiple target objects with a hierarchical relationship, the pixel blocks covered by the upper-level target object in the video frame include the pixel blocks covered by the lower-level target object in the video frame, such as Figure 2B As shown in the target object B1 and the target object B2, it can be seen that the pixel blocks covered by the target object B1 include the pixel blocks covered by the target object B2, so the target object B1 is the upper-level target object of the target object B2.

[0099] If the target object in the first video frame includes multiple target objects in a hierarchical relationship, a specified scale corresponding to the target object at each level can be determined among the multiple different scales, wherein a motion vector corresponding to a pixel block covered by the target object at the specified level exists in the video to be processed at the specified scale, and the specified scale is the smallest among the multiple different scales; and the motion vector corresponding to the pixel block covered by the target object at the specified level in the video to be processed at the specified scale is: the specified motion vector corresponding to the target object at the specified level.

[0100] For example, Figure 2C Shown are target objects C1, C2, and C3 obtained after target detection on the first video frame, wherein the relationship between the target objects is: the upper-level target object of target object C3 is target object C2, and the upper-level target object of target object C2 is target object C1. In terms of scale, scale 1 is smaller than scale 2, and scale 2 is smaller than scale 3. It can be seen that the pixel blocks covered by the same target object in video frames of different scales are different. Therefore, there may be multiple motion vectors corresponding to the pixel blocks covered by the same target object in video frames of different scales. For example, in video frames of scales 1, 2, and 3, there are motion vectors corresponding to the pixel blocks covered by target object C1. It may also be that the target object does not have a motion vector corresponding to the pixel blocks covered by the target object in video frames of smaller scales. For example, for target object C2, in the video frame of scale 1, since the pixel blocks covered by target object C2 do not match the pixel blocks divided by the video frame of scale 1, there is no designated motion vector corresponding to target object C2 in the video frame of scale 1. However, after adjusting the scale of the video frame to scale 2, the designated motion vector corresponding to target object C2 can be obtained. In summary, in the case of Figure 2C In the video frame shown, it can be determined that the designated scale of the target object C1 is scale 1 , the designated scale of the target object C2 is scale 2 , and the designated scale of the target object C3 is scale 3 .

[0101] In the embodiments of this specification, Figure 1 The position of the target object in the second video frame is determined according to the position of the target object in the first video frame and the motion vector corresponding to the second video frame located after the first video frame in the video to be processed as shown in step S104. Figure 3 As shown, this is achieved through the following steps:

[0102] S200: selecting a current scale in order from small to large according to the different scales, and determining a target object in the first video frame with the selected current scale as a designated scale as the current target object.

[0103] S202: Determine the position of the current target object in the second video frame of the current scale according to the position of the current target object in the first video frame of the current scale and the designated motion vector corresponding to the current target object.

[0104] S204: Determine whether there is an unselected scale.

[0105] If yes, execute step S200; otherwise, execute step S206.

[0106] S206: Restoring the second video frame with the largest scale to a second video frame with the original scale at the different scales; wherein the original scale includes the acquired scale of the video to be processed; and determining positions of target objects at all levels in the first video frame in the second video frame with the largest scale based on positions of the target objects at all levels in the first video frame in the second video frame with the original scale.

[0107] In this step, when it is determined that there is no unselected scale, it means that the position of the target object in the second video frame of each scale among the multiple different scales has been determined. Since there is a hierarchical relationship between the target objects at each level, each time the position of the current target object in the second video frame of the corresponding specified scale is determined, the position of the target object of the previous level of the current target object in the second video frame has been determined. Therefore, when the current scale selected from the multiple different scales is the maximum scale, the current target object with the maximum scale as the specified scale is the last level target object among the target objects at each level. Determining the position of the current target object in the second video frame of the maximum scale means determining the position of the target objects at each level in the second video frame of the maximum scale. By restoring the second video frame of the maximum scale to the second video frame of the original scale, the positions of the target objects at each level in the second video frame of the original scale are determined.

[0108] For example, Figure 4 As shown, Figure 2CThe positions of target objects at each level in the first and second video frames at the corresponding specified scales are shown. It can be seen that when the current scale selected is scale 1, the position of target object C1 in the second video frame at scale 1 can be determined based on the position of target object C1 at scale 1 in the first video frame at scale 1 and the specified motion vector of target object C1. However, since the specified scales of target object C1's subordinate target objects, target objects C2 and target objects C3, are not scale 1, the displacements of target objects C2 and target objects C3 in the second video frame at scale 1 cannot be determined. In this case, the positions of target objects C2 and target objects C3 relative to target object C1 in the second video frame at scale 1 remain unchanged. Similarly, in the second video frame at scale 3, the positions of target objects C1, target objects C2, and target objects C3 can be determined. In this case, restoring the second video frame at scale 3 to its original scale can determine the positions of target objects C1, target objects C2, and target objects C3 in the second video frame at the original scale.

[0109] In the embodiment of the present specification, as shown in step S202 of FIG2 , when determining the position of the current target object in the second video frame of the current scale according to the position of the current target object in the first video frame of the current scale and the designated motion vector corresponding to the current target object, it is necessary to consider the position of the previous level target object of the current target object in the second video frame and the influence of the position of the current target object in the second video frame, such as Figure 5 As shown, it is determined by the following steps.

[0110] S300: Determine whether the current target object has a target object at a higher level. If yes, execute step S304; otherwise, execute step S302.

[0111] Since the displacements of target objects in a hierarchical relationship between two video frames are correlated, that is, the displacement of a lower-level target object is constrained by the displacement of the upper-level target object, it is possible to determine whether the position of the current target object in the next video frame (i.e., the second video frame) is constrained by the position of the upper-level target object in the second video frame by determining whether the current target object has an upper-level target object.

[0112] S302: Determine the position of the current target object in the second video frame of the current scale according to the position of the current target object in the first video frame of the current scale and the designated motion vector corresponding to the current target object.

[0113] If the current target object has no previous-level target object, it means that the position of the current target object in the second video frame is not restricted by other target objects. Therefore, the position of the current target object in the second video frame at the current scale can be determined only based on the position of the current target in the first video frame at the current scale and the specified motion vector corresponding to the current target object.

[0114] S304: Mapping the position of the previous-level target object of the current target object in the corresponding second video frame of the specified scale to the second video frame of the current scale corresponding to the current target object to determine the position of the previous-level target object of the current target object in the second video frame of the current scale.

[0115] If the current target object has a previous level target object, it means that the position of the current target object in the second video frame can be determined according to the position of the previous level target object of the current target object in the second video frame at the current scale.

[0116] S306: Determine a relative motion vector corresponding to the current target object according to the designated motion vector corresponding to the current target object and the designated motion vector corresponding to the target object one level above the current target object.

[0117] The relative motion vector corresponding to the current target object represents the displacement of the current target object within the previous level target object from the current video frame of the current scale to the next video frame of the current scale, that is, the relative motion vector corresponding to the current target object represents the displacement of the current target object relative to the previous level target object.

[0118] S308: Determine the position of the current target object in the second video frame at the current scale according to the position of the previous level target object of the current target object in the second video frame at the current scale and the relative motion vector corresponding to the current target object.

[0119] The position of the current target object in the second video frame at the current scale is determined by combining the position of the previous target object of the current target object in the second video frame at the current scale and the displacement of the current target object relative to the previous target object.

[0120] For example, Figure 4 For example, if scale 2 is the current scale, the current target object in the first video frame with scale 2 as the current scale is target object C2. Since target object C2 has a previous-level target object C1, it is necessary to first map the target object C1 at its corresponding specified scale, that is, the position in the second video frame at scale 1, to scale 2, as shown in the following example: Figure 6A Then, as Figure 6BAs shown, based on the designated motion vector VM1 of the target object C1 and the designated motion vector VM2 of the target object C2, the relative motion vector VM3 of the target object C2 is determined to characterize the relative displacement of the target object C2 from the position in the first video frame (shown by the dotted line) to the position in the second video frame (shown by the solid line) relative to the target object C1. Based on the position of the target object C1 in the second video frame of scale 2 (shown by the solid line) and the relative motion vector VM3 of the target object C2, the position of the target object C2 in the second video frame of scale 2 (shown by the solid line) is determined. It should be noted that the designated motion vector corresponding to the target object can characterize the direction and magnitude of the displacement of the target object between the two video frames. Figure 6B The specific size and direction of the motion vector marked in needs to be determined according to the specific application scenario.

[0121] In the embodiments of this specification, Figure 1 When determining the position of the target object in the second video frame as shown in step S104, it is also necessary to determine the adjacent relationship between the second video frame and the first video frame in the video to be processed, such as Figure 7 As shown, it is determined by the following steps.

[0122] S400: Determine whether the second video frame is adjacent to the first video frame. If yes, execute step S402; otherwise, execute step S404.

[0123] S402: Determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed.

[0124] If the second video frame is adjacent to the first video frame, since the target object has been determined in the first video frame through the target detection model, the position of the target object in the second video frame can be directly determined based on the position of the target object in the first video frame and the motion vector corresponding to the second video frame located after the first video frame.

[0125] Optionally, for the case where the second video frame is adjacent to the first video frame, the second video frame may be located before the first video frame in the video to be processed. At this time, depending on the specific application scenario, it can be determined whether the position of the target object in the second video frame is determined based on the position of the target object in the previous video frame of the second video frame and the motion vector corresponding to the second video frame, or based on the position of the target object in the first video frame located after the second video frame and the motion vector corresponding to the second video frame.

[0126] S404: Determine the position of the target object in the second video frame based on the position of the target object in the video frame previous to the second video frame and the motion vector corresponding to the second video frame; wherein the position of the target object in the video frame previous to the second video frame is determined based on the first video frame and the motion vector corresponding to the video frame previous to the second video frame.

[0127] If the second video frame is not adjacent to the first video frame, it means that the two video frames before and after the second video frame are not the first video frame. At this time, the position of the target object in the second video frame can be determined based on the position of the target object in the video frame before the second video frame and the motion vector corresponding to the second video frame.

[0128] For example, Figure 8 In the video to be processed shown, video frames 1 and 5 are considered first video frames, and video frames 2 to 4, excluding the first video frame, are all second video frames. For video frame 2, since the previous video frame is video frame 1, the target object's position in video frame 2 can be determined based on the target object's position in video frame 1 and the motion vector corresponding to video frame 2. For video frame 4, since the next video frame is video frame 5, the target object's position in video frame 4 can be determined based on the target object's position in video frame 5 and the motion vector corresponding to video frame 4. For video frame 3, since the previous and next video frames are not the first video frame, that is, video frame 3 is not adjacent to the first video frame, the target object's position in video frame 3 can be determined based on the target object's position in video frame 2 and the motion vector corresponding to video frame 3. The target object's position in video frame 2 is determined based on the target object's position in video frame 1 and the motion vector corresponding to video frame 2.

[0129] The above is a target tracking method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding target tracking device, such as Figure 9 shown. Figure 9 A schematic diagram of a target tracking device provided in this manual specifically includes:

[0130] The video frame determination module 500 is configured to obtain a video to be processed and determine a first video frame and a second video frame in the video to be processed;

[0131] a target object determination module 502, configured to perform target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame;

[0132] A motion vector determination module 504 is configured to determine a motion vector corresponding to each video frame in the video to be processed;

[0133] a position determination module 506, configured to determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed;

[0134] The tracking module 508 is configured to track the target object according to the position of the target object in the first video frame and the position of the target object in the second video frame.

[0135] Optionally, the video frame determination module 500 is specifically used to determine the frequency of frame extraction based on the frame rate of the video to be processed and the frame rate of the pre-trained target detection model; according to the frequency, the video to be processed is extracted, and the extracted video frame is used as the first video frame, and the video frames other than the first video frame are used as the second video frame.

[0136] Optionally, the motion vector determination module 504 is specifically used to perform multi-scale adjustment on the video to be processed to obtain multiple videos to be processed of different scales; for each scale of the video to be processed, segment each video frame in the video to be processed of the scale to obtain pixel blocks of each video frame of the scale; and obtain the motion vector corresponding to each pixel block in each video frame of the scale as the motion vector corresponding to the video frame of the scale; the motion vector corresponding to a pixel block in a video frame is used to represent the displacement of the pixel block from the video frame to the next video frame.

[0137] Optionally, the target object in the first video frame includes multiple target objects in a hierarchical relationship; wherein the pixel blocks covered by the upper-level target object in the video frame include the pixel blocks covered by the lower-level target object in the video frame; for each level of target object, a specified scale corresponding to the target object at that level is determined among the multiple different scales, wherein a motion vector corresponding to the pixel blocks covered by the target object at that level exists in the video to be processed at the specified scale, and the specified scale is the smallest among the multiple different scales; the motion vector corresponding to the pixel blocks covered by the target object at that level in the video to be processed at the specified scale is: the specified motion vector corresponding to the target object at that level;

[0138] Optionally, the position determination module 506 is specifically configured to select the current scale in order from small to large according to the different scales, and determine the target object in the first video frame with the selected current scale as the specified scale as the current target object; determine the position of the current target object in the second video frame with the current scale according to the position of the current target object in the first video frame at the current scale and the specified motion vector corresponding to the current target object; if there is an unselected scale, continue to select the current scale in order from small to large according to the different scales, and continue to determine the position of the target object in the first video frame with the selected current scale as the specified scale in the second video frame until there is no unselected scale.

[0139] Optionally, the position determination module 506 is specifically used to, if the current target object does not have a previous-level target object, determine the position of the current target object in the second video frame of the current scale according to the position of the current target object in the first video frame of the current scale and the specified motion vector corresponding to the current target object; if the current target object has a previous-level target object, map the position of the previous-level target object of the current target object in the second video frame of the corresponding specified scale to the second video frame of the current scale corresponding to the current target object to determine the position of the previous-level target object of the current target object in the second video frame of the current scale; determine the relative motion vector corresponding to the current target object according to the specified motion vector corresponding to the current target object and the specified motion vector corresponding to the previous-level target object of the current target object; and determine the position of the current target object in the second video frame of the current scale according to the position of the previous-level target object of the current target object in the second video frame of the current scale and the relative motion vector corresponding to the current target object.

[0140] Optionally, the position determination module 506 is further configured to, when no unselected scale exists, restore the second video frame with the largest scale to a second video frame with the original scale among the different scales; wherein the original scale includes the acquired scale of the video to be processed; and determine, based on the positions of the target objects at all levels in the first video frame in the second video frame with the largest scale, the positions of the target objects at all levels in the second video frame with the original scale.

[0141] Optionally, the position determination module 506 is specifically configured to, if the second video frame is adjacent to the first video frame, determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed;

[0142] Optionally, if the second video frame is not adjacent to the first video frame, the position determination module 506 is further used to determine the position of the target object in the second video frame based on the position of the target object in the previous video frame of the second video frame and the motion vector corresponding to the second video frame; wherein, the position of the target object in the previous video frame of the second video frame is determined based on the position of the target object in the first video frame and the motion vector corresponding to the previous video frame of the second video frame.

[0143] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 Provided target tracking method.

[0144] This manual also provides Figure 10 The schematic structure diagram of the electronic device shown in FIG. Figure 10 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0145] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0146] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0147] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0148] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0149] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0153] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0154] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0155] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0156] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0157] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0159] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0160] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations may be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A target tracking method, characterized in that: include: Obtaining a video to be processed, and determining a first video frame and a second video frame in the video to be processed; Performing target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame; and determining a motion vector corresponding to each video frame in the video to be processed; determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed; Tracking the target object according to a position of the target object in the first video frame and a position of the target object in the second video frame; Determining the motion vector corresponding to each video frame in the video to be processed specifically includes: Performing multi-scale adjustment on the video to be processed to obtain multiple videos to be processed at different scales; For each scale of the video to be processed, each video frame in the video to be processed at that scale is segmented to obtain pixel blocks of each video frame at that scale; and a motion vector corresponding to each pixel block in each video frame at that scale is obtained as the motion vector corresponding to the video frame at that scale; the motion vector corresponding to a pixel block in a video frame is used to represent the displacement of the pixel block from the video frame to the next video frame; the target object in the first video frame includes multiple target objects in a superior-subordinate relationship; wherein the pixel block covered by the upper-level target object in the video frame includes the pixel block covered by the lower-level target object in the video frame; For each level of target object, determine a specified scale corresponding to the target object at that level among the multiple different scales, wherein a motion vector corresponding to a pixel block covered by the target object at that level exists in the video to be processed at the specified scale, and the specified scale is the smallest among the multiple different scales; the motion vector corresponding to the pixel block covered by the target object at that level in the video to be processed at the specified scale is: the specified motion vector corresponding to the target object at that level; determine the position of the target object in the second video frame based on the position of the target object in the first video frame and the motion vector corresponding to the second video frame located after the first video frame in the video to be processed, specifically comprising: selecting a current scale in order of the different scales from small to large, and determining the target object in the first video frame with the selected current scale as the specified scale as the current target object; Determining a position of the current target object in a second video frame at the current scale according to the position of the current target object in the first video frame at the current scale and a designated motion vector corresponding to the current target object; If there is an unselected scale, the current scale is selected in order from the smallest to the largest scale, and the position of the target object in the first video frame with the selected current scale as the specified scale is determined in the second video frame until there is no unselected scale.

2. The method according to claim 1, wherein Determining a first video frame and a second video frame in the video to be processed specifically includes: Determining a frame extraction frequency based on the frame rate of the video to be processed and the frame rate of the pre-trained object detection model; Frame extraction is performed on the video to be processed according to the frequency, the extracted video frames are used as first video frames, and video frames other than the first video frames are used as second video frames.

3. The method according to claim 1, wherein Determining, according to the position of the current target object in the first video frame at the current scale and the specified motion vector corresponding to the current target object, the position of the current target object in the second video frame at the current scale, specifically includes: If the current target object does not have a previous-level target object, determining a position of the current target object in a second video frame at the current scale according to a position of the current target object in a first video frame at the current scale and a designated motion vector corresponding to the current target object; If the current target object has a parent target object, mapping the position of the parent target object of the current target object in the corresponding second video frame of the specified scale to the second video frame of the current scale corresponding to the current target object to determine the position of the parent target object of the current target object in the second video frame of the current scale; Determining a relative motion vector corresponding to the current target object according to a designated motion vector corresponding to the current target object and a designated motion vector corresponding to a target object one level above the current target object; The position of the current target object in the second video frame at the current scale is determined according to the position of the previous level target object of the current target object in the second video frame at the current scale and the relative motion vector corresponding to the current target object.

4. The method according to claim 1, wherein The method further comprises: When there is no unselected scale, among the different scales, restoring the second video frame with the largest scale to a second video frame with an original scale; wherein the original scale includes the acquired scale of the video to be processed; Positions of target objects at all levels in the first video frame in the second video frame with the largest scale are determined according to the positions of target objects at all levels in the first video frame in the second video frame with the original scale.

5. The method according to claim 1, wherein Determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed specifically includes: If the second video frame is adjacent to the first video frame, determining the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed; If the second video frame is not adjacent to the first video frame, the method further includes: Determine the position of the target object in the second video frame based on the position of the target object in the video frame previous to the second video frame and the motion vector corresponding to the second video frame; wherein the position of the target object in the video frame previous to the second video frame is determined based on the position of the target object in the first video frame and the motion vector corresponding to the video frame previous to the second video frame.

6. A target tracking device, characterized in that: include: A video frame determination module is used to obtain a video to be processed and determine a first video frame and a second video frame in the video to be processed; a target object determination module, configured to perform target detection on the first video frame using a pre-trained target detection model to determine a target object in the first video frame and a position of the target object in the first video frame; A motion vector determination module, configured to determine a motion vector corresponding to each video frame in the video to be processed; a position determination module, configured to determine the position of the target object in the second video frame according to the position of the target object in the first video frame and a motion vector corresponding to a second video frame located after the first video frame in the video to be processed; a tracking module, configured to track the target object according to a position of the target object in the first video frame and a position of the target object in the second video frame; Determining the motion vector corresponding to each video frame in the video to be processed specifically includes: Performing multi-scale adjustment on the video to be processed to obtain multiple videos to be processed at different scales; For each scale of the video to be processed, each video frame in the video to be processed at that scale is segmented to obtain pixel blocks of each video frame at that scale; and a motion vector corresponding to each pixel block in each video frame at that scale is obtained as the motion vector corresponding to the video frame at that scale; the motion vector corresponding to a pixel block in a video frame is used to represent the displacement of the pixel block from the video frame to the next video frame; the target object in the first video frame includes multiple target objects in a superior-subordinate relationship; wherein the pixel block covered by the upper-level target object in the video frame includes the pixel block covered by the lower-level target object in the video frame; For each level of target object, determine a specified scale corresponding to the target object at that level among the multiple different scales, wherein a motion vector corresponding to a pixel block covered by the target object at that level exists in the video to be processed at the specified scale, and the specified scale is the smallest among the multiple different scales; the motion vector corresponding to the pixel block covered by the target object at that level in the video to be processed at the specified scale is: the specified motion vector corresponding to the target object at that level; determine the position of the target object in the second video frame based on the position of the target object in the first video frame and the motion vector corresponding to the second video frame located after the first video frame in the video to be processed, specifically comprising: selecting a current scale in order of the different scales from small to large, and determining the target object in the first video frame with the selected current scale as the specified scale as the current target object; Determining a position of the current target object in a second video frame at the current scale according to the position of the current target object in the first video frame at the current scale and a designated motion vector corresponding to the current target object; If there is an unselected scale, the current scale is selected in order from the smallest to the largest scale, and the position of the target object in the first video frame with the selected current scale as the specified scale is determined in the second video frame until there is no unselected scale.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images