Target tracking method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a target tracking method and device, electronic equipment and a readable storage medium. According to the embodiment of the invention, after a video is obtained, target detection is carried out on each image in the video, a to-be-matched object in each image is determined, and a corresponding relation between each to-be-matched object and each reference object in the same image is determined according to an overlapping area of each to-be-matched object and each reference object in the same image; and determining the object identifier of each to-be-matched object in each image according to the reference object identifier of the reference object and the corresponding relationship between each to-be-matched object and each reference object in the same image, so as to perform target tracking on the to-be-matched object according to the position of the to-be-matched object in each image. In the embodiment of the invention, the object identification of each to-be-matched object is determined in each image in a mode of performing target detection on the to-be-detected image and matching the to-be-detected image with the reference object, so that the consumption of computing resources can be reduced while target tracking is realized, and the target tracking efficiency is improved at the same time.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a target tracking method, device, electronic device and readable storage medium. Background Art

[0002] In the field of computer vision, object tracking is a crucial method for determining the location, category, and other key information about objects of interest in an image. However, existing object tracking algorithms are complex, consuming significant computing resources and resulting in low object tracking efficiency. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a target tracking method, device, electronic device and readable storage medium, which is used to determine the object identification of each object to be matched in each image by performing target detection on the image to be detected and matching it with a reference object. It can reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking.

[0004] In a first aspect, an embodiment of the present invention provides a target tracking method, the method comprising:

[0005] Acquire a target video, where the target video is a sequence consisting of a plurality of target images, each of which includes a reference object;

[0006] Performing target detection on at least one of the target images to determine an object to be matched in each of the target images;

[0007] determining a correspondence between each of the objects to be matched and each of the reference objects in the same target image according to a first overlapping area between each of the objects to be matched and each of the reference objects in the same target image;

[0008] The object identifier of each of the reference objects is determined in each of the target images according to the reference object identifier and the corresponding relationship, so as to track the object to be matched according to the position of the object to be matched in each of the target images.

[0009] Optionally, determining the correspondence between each of the to-be-matched objects and each of the reference objects in the same target image according to a first overlapping area between each of the to-be-matched objects and each of the reference objects in the same target image includes:

[0010] Determine each of the target images as an image to be detected;

[0011] Determining, in the image to be detected, a primary reference object to be matched with each of the objects to be matched, according to the first overlapping area between each of the objects to be matched and each of the reference objects in the image to be detected;

[0012] Determining, in the image to be detected, a secondary reference object to be matched by each of the objects to be matched according to a first area ratio between each of the objects to be matched and each of the primary reference objects to be matched;

[0013] The target reference object matched by each to-be-matched object in the image to be detected is determined according to the distance between each to-be-matched object and the image acquisition device, so as to determine the corresponding relationship according to the matching relationship between each to-be-matched object and each reference object.

[0014] Optionally, determining, in the image to be detected, a primary reference object to which each of the objects to be matched matches based on the first overlapping area between each of the objects to be matched and each of the reference objects in the image to be detected includes:

[0015] For each of the objects to be matched, the matching primary reference object is determined in the image to be detected according to the ratio of the corresponding first overlapping area to the second area of ​​the object to be matched.

[0016] Optionally, determining the corresponding relationship according to the matching relationship between each of the to-be-matched objects and each of the reference objects includes:

[0017] In response to the matching relationship representation that at least one of the reference objects matches a plurality of the objects to be matched, determining each of the reference objects that matches the plurality of objects to be matched as a reference object to be matched;

[0018] determining, in the image to be detected, a primary object matched by the reference object to be matched based on a second overlapping area between each of the objects to be matched and a standard detection frame, wherein the standard detection frame is an object detection frame of the object to be matched whose relative position to the reference object to be matched remains unchanged;

[0019] determining a target object to be matched by the reference object to be matched in the image to be detected according to the confidence of the object detection frame of each of the first-level objects in the image to be detected;

[0020] The corresponding relationship is determined according to each of the reference objects to be matched and the target object to be matched.

[0021] Optionally, determining the primary object matched by the reference object to be matched in the image to be detected according to the second overlapping area between each of the objects to be matched and the standard detection frame includes:

[0022] The primary object matching the reference object to be matched is determined in the image to be detected according to the ratio of the second overlapping area between the same object to be matched that matches the reference object to be matched and the standard detection frame to a third area of ​​the same object to be matched.

[0023] Optionally, the standard detection frame is determined by:

[0024] Determine a candidate object in each of the images to be detected, wherein the candidate object is the object to be matched that is located within a reference object detection frame of the reference object to be matched;

[0025] The standard detection frame is determined from the object detection frames according to the confidence of the object detection frame of each candidate object.

[0026] Optionally, the position of the standard detection frame is determined by:

[0027] Determining the image to be detected where the standard detection frame is located as a standard image;

[0028] Determining the relative position of the standard detection frame and the reference object to be matched according to the position of the standard detection frame in the standard image and the position of the reference object to be matched in the standard image;

[0029] The position of the standard detection frame in each of the images to be detected is determined according to the position of the reference object to be matched in each of the images to be detected and the relative position.

[0030] Optionally, determining the corresponding relationship according to the matching relationship between each of the to-be-matched objects and each of the reference objects includes:

[0031] In response to the matching relationship representation that at least one of the reference objects matches one of the objects to be matched, determining the reference object that matches one of the objects to be matched as a matched reference object;

[0032] The corresponding relationship is determined according to the matched reference object and the matched object to be matched.

[0033] Optionally, the method further includes:

[0034] In response to at least one of the reference objects in the target image not detecting the corresponding object to be matched, determining the target image not detecting the corresponding object to be matched as the current image;

[0035] The target position of the object to be matched is determined based on at least one of a first position of the object to be matched and a second position of the object to be matched, wherein the first position is the position of the object to be matched detected in the target image sequentially before the current image, the second position is the position of the object to be matched detected in the target image sequentially after the current image, and the target position is the position of the object to be matched in the current image.

[0036] Optionally, determining the target position of the object to be matched according to at least one of the first position of the object to be matched and the second position of the object to be matched includes:

[0037] Performing linear fitting based on at least one of a plurality of continuous first positions and a plurality of continuous second positions to obtain a position fitting function of the object to be matched;

[0038] The target position is determined according to the position fitting function and a timestamp of the current image.

[0039] Optionally, performing linear fitting based on at least one of a plurality of consecutive first positions and a plurality of consecutive second positions to obtain a position fitting function of the object to be matched includes:

[0040] Performing forward Kalman filtering based on a plurality of consecutive first positions to obtain the position fitting function; or

[0041] Backward Kalman filtering is performed based on a plurality of continuous second positions to obtain the position fitting function.

[0042] Optionally, determining the target position of the object to be matched according to at least one of the first position of the object to be matched and the second position of the object to be matched includes:

[0043] Determine a first target position from each of the first positions, where the first target position is the first position of the object to be matched detected in the target image with a timestamp closest to that of the current image;

[0044] Determine a second target position from each of the second positions, where the second target position is the second position of the object to be matched detected in the target image with a timestamp closest to that of the current image;

[0045] The target position is determined by interpolating the target position based on the first target position and the second target position.

[0046] In a second aspect, an embodiment of the present invention provides a target tracking device, the device comprising:

[0047] a video acquisition unit, configured to acquire a target video, wherein the target video is a sequence composed of a plurality of target images, each of which includes a reference object;

[0048] a target detection unit, configured to perform target detection on at least one of the target images and determine an object to be matched in each of the target images;

[0049] a relationship determining unit, configured to determine a corresponding relationship between each of the to-be-matched objects and each of the reference objects in the same target image according to a first overlapping area between each of the to-be-matched objects and each of the reference objects in the same target image;

[0050] The identification determination unit is used to determine the object identification of each of the objects to be matched in each of the target images according to the reference object identification of each of the reference objects and the corresponding relationship, so as to track the objects to be matched according to their positions in each of the target images.

[0051] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0052] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the first aspects is implemented.

[0053] After acquiring the video, the embodiment of the present invention performs target detection on each image in the video, determines the object to be matched in each image, and determines the correspondence between each object to be matched and each reference object in the image based on the overlapping area of ​​each object to be matched and each reference object in the same image, and then determines the object identification of each object to be matched in each image based on the reference object identification of the reference object and the correspondence between each object to be matched and each reference object in the same image, so as to track the object to be matched according to its position in each image. In the embodiment of the present invention, by performing target detection on the image to be detected and matching the reference object to determine the object identification of each object to be matched in each image, it is possible to reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0055] Figure 1is a schematic diagram of the hardware system architecture of an embodiment of the present invention;

[0056] Figure 2 is a flow chart of a target tracking method according to a first embodiment of the present invention;

[0057] Figure 3 This is a flow chart of determining the correspondence between each to-be-matched object and each reference object in the same target image in an optional implementation of the first embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a reference object and an object to be matched in a target image according to an embodiment of the present invention;

[0059] Figure 5 This is a flow chart of determining the correspondence between each to-be-matched object and each reference object in the same target image in an optional implementation of the first embodiment of the present invention;

[0060] Figure 6 is a schematic diagram of the relative positions of a reference object and a standard detection frame according to an embodiment of the present invention;

[0061] Figure 7 is another schematic diagram of the relative positions of the reference object and the standard detection frame according to an embodiment of the present invention;

[0062] Figure 8 is a flow chart of a target tracking method according to a second embodiment of the present invention;

[0063] Figure 9 is a schematic diagram of determining the position of the object to be matched in the current image according to the position of the object to be matched corresponding to the reference object in the target image in an embodiment of the present invention;

[0064] Figure 10 is a schematic diagram of a target tracking device according to a third embodiment of the present invention;

[0065] Figure 11 FIG. 4 is a schematic diagram of an electronic device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0067] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0068] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0069] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0070] Where the solutions described in this specification and in the examples involve the processing of personal information, such processing will be conducted with a legitimate basis (e.g., with the consent of the personal information subject or as necessary for the performance of a contract) and only within the prescribed or agreed scope. A user's refusal to process personal information other than that required for basic functions will not affect the user's use of these basic functions.

[0071] In the field of computer vision, target tracking is one of the important means to find key information such as the object identity, location, and category of all objects of interest in an image. In daily life, target tracking is widely used in various fields, such as sports event broadcasting, security systems, and transportation systems. In the existing technology, target tracking is usually achieved through tracking algorithms based on optical flow, such as KLT (Kanade-Lucas-Tomasi), end-to-end tracking algorithms based on deep learning, such as CenterTrack, and tracking algorithms based on feature matching, such as SIFT (Scale Invariant Feature Transferform) and SURF (Speeded Up Robust Features). However, the above-mentioned target tracking algorithms track each object in the image or video separately, which leads to high complexity, high consumption of computing resources, and low target tracking efficiency.

[0072] In existing technologies, target tracking can be achieved using relatively simple target tracking algorithms such as SORT (Simple Online Realtime Tracking). However, SORT is prone to tracking drift, where different object identifiers are detected for the same object in different image frames. This is particularly true when the objects to be detected are relatively small in the image, resulting in lower target tracking accuracy.

[0073] Figure 1 Schematic diagram of the hardware system architecture of an embodiment of the present invention. Figure 1As shown, the embodiment of the present invention includes at least one image acquisition device 11 and at least one image processing device 12. Figure 1 An image acquisition device 11 and an image processing device 12 are taken as an example for description.

[0074] The image acquisition device 11 of the embodiment of the present invention can be any existing imaging device, such as a video camera, a camera, or an intelligent terminal with a camera function (such as a smartphone, tablet computer, personal computer, or driving recorder). Depending on actual needs, the image acquisition device 11 can be a fixed-position imaging device or a movable imaging device, which is not limited in the embodiment of the present invention.

[0075] The image processing device 12 of the embodiment of the present invention can be a server or an intelligent terminal with image processing capabilities. In some embodiments, the image processing device 12 is a server, and the image processing device 12 can be a single server or a server group. The server group can be centralized or distributed. Optionally, the server can establish a communication connection with the image acquisition device 11 through a network, and access information and / or data stored in the image acquisition device 11, the storage device of the shared vehicle 13, the database, or any combination thereof via the network; optionally, the server can be implemented on a cloud platform, such as a private cloud, a public cloud, a hybrid cloud, a community cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof.

[0076] In some embodiments, the image processing device 12 is an intelligent terminal with image processing capabilities, such as a smartphone, tablet computer, personal computer, onboard computer (also known as car computer, driving computer, computer control module ECU), etc. Optionally, the image acquisition device 11 and the image processing device 12 can be an integrated intelligent terminal, that is, the image acquisition device 11 and the image processing device 12 are the same intelligent terminal; optionally, the image acquisition device 11 and the image processing device 12 can also be split intelligent terminals, that is, the image acquisition device 11 and the image processing device 12 are different intelligent terminals. When the image acquisition device 11 and the image processing device 12 are different intelligent terminals, the image acquisition device 11 and the image processing device 12 can establish a short-range communication connection, such as establishing a short-range communication connection through the Bluetooth transmission protocol, Zigbee, etc., or establish a long-range communication connection through a network, such as a Global System for Mobile Communications (GSM) module or a General Packet Radio Service (GPRS) module.

[0077] In an embodiment of the present invention, the image acquisition device 11 captures a video and sends the video to the image processing device 12. After acquiring the video, the image processing device 12 determines a sequence of images (i.e., target images) in the video including reference objects as a target video, performs target detection on each target image, determines the to-be-matched object in each target image, and then determines the correspondence between each to-be-matched object and each reference object in the same target image based on the overlapping area (i.e., the first overlapping area) of each to-be-matched object and each reference object in the same target image. Thus, the object identifier of each to-be-matched object is determined in each target image based on the reference object identifier of each reference object and the corresponding relationship between each to-be-matched object and each reference object in the same target image, so as to track the to-be-matched object according to its position in the target image.

[0078] In an optional implementation of the embodiment of the present invention, the image processing device 12 determines each target image as an image to be detected, and determines the primary reference object matched by each object to be matched in the image to be detected according to the overlapping area between each object to be matched and each reference object in the image to be detected, and then determines the secondary reference object matched by each object to be matched in the image to be detected according to the area ratio (i.e., the first area ratio) between each object to be matched and each primary reference object matched in each image to be detected, thereby determining the target reference object matched by each object to be matched in the image to be detected according to the distance between each secondary reference object matched by each object to be matched and the image acquisition device 11, so as to determine the corresponding relationship between each object to be matched and each reference object according to the matching relationship between each object to be matched and each reference object.

[0079] In an optional implementation method of an embodiment of the present invention, for each object to be matched, the image processing device 12 determines the ratio of the overlapping area between the object to be matched and each reference object in the image to be detected to the area of ​​the object to be matched (that is, the second area ratio), and determines the primary reference object that matches the object to be matched in the image to be detected based on the ratio of the overlapping area between the object to be matched and each reference object and the area of ​​the object to be matched.

[0080] In an optional implementation of an embodiment of the present invention, when the matching relationship representation indicates that there is at least one reference object that matches multiple objects to be matched, the image processing device 12 will determine each reference object that matches the multiple objects to be matched as a reference object to be matched, and determine the first-level object that matches the reference object to be matched in the image to be detected based on the overlapping area (i.e., the second overlapping area) between each object to be matched and the standard detection frame matched by each reference object to be matched, wherein the standard detection frame is an object detection frame of the object to be matched whose relative position with the reference object to be matched remains unchanged, and then determine the target object that matches the reference object to be matched in the image to be detected based on the confidence of the object detection frame of each first-level object in the image to be detected, thereby determining the correspondence between the object to be matched and the reference object based on each reference object to be matched and the matched target object.

[0081] In an optional implementation of an embodiment of the present invention, the image processing device 12 will determine the primary object that matches the reference object to be matched in the image to be detected based on the ratio of the overlapping area between the same object to be matched and the standard detection frame that matches the reference object to be matched and the area of ​​the same object to be matched (that is, the third area ratio).

[0082] In an optional implementation of an embodiment of the present invention, the image processing device 12 determines the object to be matched within the reference object detection frame of the reference object to be matched as a candidate object, and determines a standard detection frame from each object detection frame based on the confidence of the object detection frame of each candidate object.

[0083] In an optional implementation of an embodiment of the present invention, the image processing device 12 will determine the image to be detected where the standard detection frame is located as the standard image, and determine the relative position of the standard detection frame and the reference object to be matched according to the position of the standard detection frame in the standard image and the position of the reference object to be matched in the standard image, thereby determining the position of the standard detection frame in each image to be detected according to the position of the reference object to be matched in each image to be detected and the relative position of the standard detection frame and the reference object to be matched.

[0084] In an optional implementation of an embodiment of the present invention, when the matching relationship represents that there is at least one reference object that matches an object to be matched, the image processing device 12 will determine the reference object that matches the object to be matched as a matched reference object, and determine the corresponding relationship between the object to be matched and the reference object based on the matched reference object and the matched object to be matched.

[0085] In an optional implementation of an embodiment of the present invention, when there is at least one reference object in the target image in which the corresponding object to be matched is not detected, the image processing device 12 will determine the target image in which the corresponding object to be matched is not detected as the current image, and determine the position of the object to be matched in the current image (i.e., the target position) based on at least one of the position of the object to be matched detected in the target image before the current image (i.e., the first position) and the position of the object to be matched detected in the target image after the current image (i.e., the second position).

[0086] In an optional implementation of an embodiment of the present invention, the image processing device 12 performs linear fitting based on at least one of multiple continuous first positions and multiple continuous second positions, obtains a position fitting function of the object to be matched, and determines the target position based on the position fitting function and the timestamp of the current image.

[0087] In an optional implementation of an embodiment of the present invention, the image processing device 12 performs forward Kalman filtering based on multiple consecutive first positions to obtain a position fitting function, or performs backward Kalman filtering based on multiple consecutive second positions to obtain a position fitting function.

[0088] In an optional implementation of an embodiment of the present invention, the image processing device 12 determines the position of the object to be matched detected in the target image closest to the timestamp of the current image from each first position (i.e., the first target position), and determines the position of the object to be matched detected in the target image closest to the timestamp of the current image from each second position (i.e., the second target position), thereby interpolating the target position based on the first target position and the second target position to determine the value of the target position.

[0089] After acquiring the video, the embodiment of the present invention performs target detection on each image in the video, determines the object to be matched in each image, and determines the correspondence between each object to be matched and each reference object in the image based on the overlapping area of ​​each object to be matched and each reference object in the same image, and then determines the object identification of each object to be matched in each image based on the reference object identification of the reference object and the correspondence between each object to be matched and each reference object in the same image, so as to track the object to be matched according to its position in each image. In the embodiment of the present invention, by performing target detection on the image to be detected and matching the reference object to determine the object identification of each object to be matched in each image, it is possible to reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking.

[0090] Figure 2 FIG. 1 is a flow chart of the target tracking method according to the first embodiment of the present invention. Figure 2As shown, the method of this embodiment includes the following steps:

[0091] Step S100: Acquire target video.

[0092] When the image capture device is turned on, it captures video within a specific range, such as a sector-shaped area centered on the image capture device's location and with a radius equal to the device's shooting distance, as the original video, and transmits the original video to the image processing device. The original video may contain segments or images that do not include the reference object. Therefore, the image processing device can perform target tracking on the original video and identify the segment containing the reference object as the target video.

[0093] In this embodiment, the reference object is an easily detectable target in the original video image. Therefore, the reference object typically occupies a large area in the original video image, or the color of the reference object is significantly different from the surrounding environment. Depending on actual needs, the reference object can be a vehicle, a ship, etc.

[0094] Optionally, the image processing device may perform target tracking on the original video based on methods such as KLT, CenterTrack, SIFT, and SURF, remove images from the original video that do not include the reference object, and identify the video segments that include the reference object as the target video. Furthermore, after performing target tracking on the original video, the image processing device may identify video segments that include the same reference object as a target video based on the reference object identifiers of the reference objects.

[0095] For example, the original video is a 5-minute video. After the image processing device performs target tracking on the original video, it tracks 5 reference objects, namely vehicle C1, vehicle C2, vehicle C3, vehicle C4, and vehicle C5. The time range for tracking vehicle C1 in the original video is 10 seconds to 2 minutes and 30 seconds, the time range for tracking vehicle C2 is 25 seconds to 1 minute and 30 seconds, the time range for tracking vehicle C3 is 30 seconds to 3 minutes and 50 seconds, the time range for tracking vehicle C4 is 3 minutes and 30 seconds to 4 minutes and 10 seconds, and the time range for tracking vehicle C5 is 2 minutes and 40 seconds to 4 minutes and 45 seconds. Then, the image processing device can respectively determine video segment 1 from 10 seconds to 2 minutes and 30 seconds, video segment 2 from 25 seconds to 1 minute and 30 seconds, video segment 3 from 30 seconds to 3 minutes and 50 seconds, video segment 4 from 3 minutes and 30 seconds to 4 minutes and 10 seconds, and video segment 5 from 2 minutes and 40 seconds to 4 minutes and 45 seconds in the original video as target videos.

[0096] Step S200: performing target detection on at least one target image to determine an object to be matched in each target image.

[0097] After determining the target video, the image processing device can perform target detection on the target image in the target video frame by frame, or extract a target image from the target video for target detection according to a predetermined period, for example, extracting one target image for target detection every four target images, and determining at least one object to be matched in each target image. In this embodiment, the object to be matched is a target in the target image that is not easy to detect relative to the reference object, so the area of ​​the object to be matched in the target image is usually small, or the color of the object to be matched is not obviously different from the surrounding environment. For example, when the reference object is a vehicle, the object to be matched can be the vehicle logo of the vehicle, etc. In this step, the image processing device only needs to detect each object to be matched in the target image, and does not need to determine the object identification of each object to be matched. Therefore, it can effectively reduce the complexity of target tracking of the object to be matched, thereby reducing the consumption of computing resources.

[0098] Step S300 : determining a correspondence between each to-be-matched object and each reference object in the same target image according to a first overlapping area between each to-be-matched object and each reference object in the same target image.

[0099] In this embodiment, the reference object and the object to be matched are positionally associated. That is, if the reference object is tracked in the target image, the target image will generally include the object to be matched corresponding to the reference object. If the reference object is not tracked in the target image, the target image will generally not include the object to be matched corresponding to the reference object. At the same time, the relative position of the object to be matched and the reference object does not change, and the object to be matched and the reference object overlap in the target image. That is, the object detection frame of the object to be matched is located within the reference object detection frame of the reference object. Therefore, in this embodiment, the image processing device can determine the corresponding relationship between each object to be matched and each reference object in each target image based on the overlapping area of ​​each object to be matched and each reference object in the same target image.

[0100] For objects to be matched of the same category, there is a one-to-one correspondence between the objects to be matched and the reference objects. For example, when the category of the objects to be matched is vehicle signs, each vehicle sign corresponds to a vehicle.

[0101] Figure 3 FIG. 1 is a flow chart of determining the correspondence between each to-be-matched object and each reference object in the same target image in an optional implementation of the first embodiment of the present invention. Figure 3 As shown, in an optional implementation of this embodiment, step S300 may include the following steps:

[0102] Step S310: Determine each target image as an image to be detected.

[0103] In this step, the image processing device may determine each target image as an image to be detected frame by frame, or may extract a target image from the target video as an image to be detected according to a predetermined period, for example, extracting one target image as an image to be detected every four target images, which is not limited in this embodiment. It is easy to understand that in this embodiment, each target image may be determined as an image to be detected in an iterative manner, or all target images may be determined as images to be detected simultaneously.

[0104] Step S320 : determining a primary reference object to be matched with each to-be-matched object in the to-be-detected image according to a first overlapping area between each to-be-matched object and each reference object in the to-be-detected image.

[0105] When outputting a detection result, the target detection algorithm may output the position parameters of the object detection frame in the image to be detected, such as the upper left corner coordinates and lower right corner coordinates of the object detection frame, or the center coordinates of the object detection frame and the width and height of the object detection frame, simultaneously with the object detection frame of the object to be matched. Similarly, when outputting a detection result, the target tracking algorithm may output the position parameters of the reference detection frame in the image to be detected, simultaneously with the reference detection frame of the reference object. Therefore, in this step, to facilitate determining the correspondence between the object to be matched and the reference object, the image processing device may determine the area of ​​the corresponding object detection frame based on the position parameters of the object detection frame of each object to be matched, and determine the area of ​​the object detection frame of each object to be matched as the area of ​​the object to be matched. At the same time, the image processing device may determine the area of ​​the corresponding reference detection frame based on the position parameters of the reference detection frame of each reference object, and determine the area of ​​the reference detection frame of each reference object as the area of ​​the reference object, thereby determining the overlapping area of ​​each object to be matched and each reference object in the image to be detected, so as to determine the primary reference object that matches each object to be matched in the image to be detected.

[0106] Optionally, the image processing device may also perform contour detection on each to-be-matched object and each reference object to obtain the contours of each to-be-matched object and each reference object, and then determine the area of ​​each to-be-matched object and the area of ​​each reference object based on the contours of each to-be-matched object and each reference object. Contour detection can be implemented using various existing methods, such as using the cv.findContours() function, and this embodiment does not impose any limitation thereto.

[0107] After respectively determining the overlapping area between each object to be matched and each reference object in the image to be detected, the image processing device can respectively determine the reference object whose overlapping area with each object to be matched is equal to the area of ​​the object to be matched as the primary reference object matching the object to be matched.

[0108] Optionally, to further reduce computing resource consumption by the image processing device, for any object to be matched, the image processing device may determine, in the image to be detected, a primary reference object that matches the object to be matched based on the ratio of the overlapping area between the object to be matched and each reference object in the image to be detected, and the area of ​​the object to be matched. Specifically, the image processing device may determine each reference object whose area ratio with each object to be matched is equal to 1 as the primary reference object that matches the object to be matched.

[0109] For example, image 1 is the image to be detected. The reference objects in image 1 include vehicles C1, C2, C3, C4, and C5, and the objects to be matched include markers M1, M2, M3, M4, M5, M6, and M7. For marker M1, the image processing device can determine the overlapping area a1 of marker M1 with vehicle C1, the overlapping area a2 of marker M1 with vehicle C2, the overlapping area a3 of marker M1 with vehicle C3, the overlapping area a4 of marker M1 with vehicle C4, and the overlapping area a5 of marker M1 with vehicle C5. Furthermore, the image processing device can determine the area ratio AR1 of overlapping area a1 to marker M1, the area ratio AR2 of overlapping area a2 to marker M1, the area ratio AR3 of overlapping area a3 to marker M1, the area ratio AR4 of overlapping area a4 to marker M1, and the area ratio AR5 of overlapping area a5 to marker M1. Since the area ratios a1 and a2 are both 1, the image processing device can determine vehicles C1 and C2 as primary reference objects for marker M1.

[0110] Step S330 : determining a secondary reference object to be matched with each to-be-matched object in the image to be detected according to the first area ratio between each to-be-matched object and the corresponding primary reference object.

[0111] After determining the primary reference objects corresponding to each object to be matched, for any object to be matched, the image processing device can determine the area ratio of the object to be matched and the corresponding primary reference objects, and determine the primary reference object whose area ratio meets the first threshold condition as the secondary reference object of the object to be matched in the image to be detected.

[0112] In this optional implementation, the first threshold condition may include the area ratio of the to-be-matched object to the corresponding primary reference object being greater than or equal to the first threshold, less than or equal to the first threshold, or falling within a first value range, where the first threshold or the first value range may be determined statistically. Specifically, the first threshold or the first value range may be determined based on the area ratios of multiple reference objects to the target objects corresponding to each reference object, where each reference object belongs to the same category and each target object belongs to the same category.

[0113] For example, when the category of the reference object is a motor vehicle and the category of the target object is a motor vehicle sign, the area of ​​the vehicle detection frame (that is, the reference object detection frame) of motor vehicles such as passenger cars (including micro passenger cars, ordinary passenger cars, mid-range passenger cars, mid-to-high-end passenger cars and high-end passenger cars, etc.), buses, trucks, off-road vehicles, dump trucks, tractors, special-purpose vehicles, etc. can be obtained, and the area of ​​the sign detection frame (that is, the object detection frame) of the motor vehicle sign of each motor vehicle can be obtained respectively to obtain the area ratio of each motor vehicle to the corresponding motor vehicle sign, so that the minimum value or maximum value of the area ratio can be determined as the first threshold, or the minimum value of the area ratio can be determined as the lower limit of the first value range, and the maximum value of the area ratio can be determined as the upper limit of the first value range.

[0114] It is easy to understand that in this optional implementation, if there is at least one object to be matched that has a one-to-one matching relationship with the primary reference object, the image processing device can directly determine the corresponding relationship between the at least one object to be matched and the primary reference object, and there is no need to execute steps S330-S380; if there is at least one object to be matched that only matches one primary reference object, but the reference object matches multiple objects to be matched, the image processing device may not execute steps S330 and S340; if there is at least one object to be matched that only matches one secondary reference object, but the reference object matches multiple objects to be matched, the image processing device may not execute step S340.

[0115] Step S340 : determining the target reference object to be matched by each to-be-matched object in the to-be-detected image according to the distance between each secondary reference object corresponding to each to-be-matched object and the image acquisition device.

[0116] After determining the secondary reference objects corresponding to each object to be matched, for any object to be matched, the image processing device can determine the distance between each secondary reference object corresponding to the object to be matched and the image acquisition device, and determine the secondary reference object closest to the image acquisition device as the target reference object for matching the object to be matched in the image to be detected.

[0117] In this optional implementation, the image processing device can determine the distance between the secondary reference object and the image acquisition device through various existing methods. Specifically, the distance between the secondary reference object and the image acquisition device can be determined using a distance measuring device, such as an infrared distance measuring device or an ultrasonic distance measuring device, or according to a predetermined image ranging algorithm. Optionally, if the intrinsic parameters, extrinsic parameters, distortion parameters, and other parameters of the image acquisition device are unknown, the image processing device can calibrate the image acquisition device based on the target image, that is, determine the intrinsic parameters, extrinsic parameters, and distortion parameters of the image acquisition device, thereby implementing the image ranging algorithm by identifying the size of the reference object in the image to be detected, the actual size of the reference object, and the intrinsic parameters, extrinsic parameters, and distortion parameters of the image acquisition device.

[0118] Figure 4 Schematic diagram of a reference object and an object to be matched in a target image according to an embodiment of the present invention. Figure 4 The image 40 shown is a target image in the target video. After performing target detection on the target image 40, the image processing device determines that the image 40 includes multiple reference objects and multiple objects to be matched, where the reference objects include motor vehicle A, motor vehicle B and motor vehicle C, and the objects to be matched include motor vehicle mark a, motor vehicle mark b, motor vehicle mark c and motor vehicle mark d.

[0119] The image processing device may calculate the area ratio of the detection frame 41 of the motor vehicle A to the detection frame 44 of the motor vehicle mark a and the area ratio of the detection frame 45 of the motor vehicle mark b to the detection frame 45 of the motor vehicle mark b, the detection frame 41 of the motor vehicle A to the detection frame 46 of the motor vehicle mark c and the area ratio of the detection frame 47 of the motor vehicle mark d to the detection frame 47 of the motor vehicle mark d, the detection frame 42 of the motor vehicle B to the detection frame 44 of the motor vehicle mark a and the area ratio of the detection frame 44 of the motor vehicle mark a, the detection frame 42 of the motor vehicle B to the detection frame 45 of the motor vehicle mark b and the detection frame 45 of the motor vehicle mark b, the detection frame 42 of the motor vehicle B to the detection frame 46 of the motor vehicle mark c and the area ratio of the detection frame 47 of the motor vehicle mark d, The area ratio of the detection frame 46 of motor vehicle B, the area ratio of the overlapping area of ​​the detection frame 42 of motor vehicle B and the detection frame 47 of motor vehicle mark d to the area ratio of the detection frame 47 of motor vehicle mark d, the area ratio of the overlapping area of ​​the detection frame 43 of motor vehicle C and the detection frame 44 of motor vehicle mark c to the area ratio of the detection frame 44 of motor vehicle mark c, the area ratio of the overlapping area of ​​the detection frame 43 of motor vehicle C and the detection frame 45 of motor vehicle mark b to the detection frame 45 of motor vehicle mark b, the area ratio of the overlapping area of ​​the detection frame 43 of motor vehicle C and the detection frame 46 of motor vehicle mark c to the detection frame 46 of motor vehicle mark c, and the area ratio of the overlapping area of ​​the detection frame 43 of motor vehicle C and the detection frame 47 of motor vehicle mark d to the detection frame 47 of motor vehicle mark d, determine that the primary reference object matched by motor vehicle mark a is motor vehicle A, the primary reference object matched by motor vehicle mark b is motor vehicle A, the primary reference object matched by motor vehicle mark c is motor vehicle B, and the primary reference object matched by motor vehicle mark 4d is motor vehicle C. Among them, the motor vehicle mark c and the motor vehicle B, and the motor vehicle mark d and the motor vehicle C are all matched one to one, so the image processing device can determine that the motor vehicle mark c corresponds to the motor vehicle B, and the motor vehicle mark d corresponds to the motor vehicle C.

[0120] Vehicle mark a matches only with vehicle A, and vehicle mark b matches only with vehicle A. However, vehicle A matches both vehicle marks a and b, so the image processing device does not need to perform the following process. Based on the area ratio of vehicle mark a's detection frame 44 to vehicle A's detection frame 41, and the area ratio of vehicle mark b's detection frame 45 to vehicle A's detection frame 41, the image processing device can determine that the secondary reference object matched by vehicle mark a is vehicle A, and that the secondary reference object matched by vehicle mark c is also vehicle A.

[0121] After determining the secondary reference objects that match motor vehicle sign a and motor vehicle sign c, for motor vehicle sign a and motor vehicle sign b, the image processing device can determine that motor vehicle A is the secondary reference object of the distance image acquisition device, and therefore it can be determined that the target reference objects that match motor vehicle sign a and motor vehicle sign b are both motor vehicle A.

[0122] After the above steps, each object to be matched will be matched with a reference object. In a possible case, when the objects to be matched and the reference objects are matched one-to-one, the image processing device can directly determine the corresponding relationship between each object to be matched and each reference object.

[0123] Figure 5 This is a flow chart of determining the correspondence between each to-be-matched object and each reference object in the same target image in an optional implementation of the first embodiment of the present invention. In another possible situation, when the to-be-matched object and the reference object are not in a one-to-one matching relationship, that is, when the same reference object matches multiple different to-be-matched objects, such as Figure 5 As shown, step S300 may further include the following steps:

[0124] Step S350 : Determine each reference object that matches the plurality of objects to be matched as a reference object to be matched.

[0125] In this step, the image processing apparatus may iteratively determine the reference objects that match the multiple objects to be matched as reference objects to be matched. It is easy to understand that in this embodiment, step S350 is performed after step S340.

[0126] Step S360 : determining the primary objects to be matched by the reference object to be matched in the image to be detected according to the second overlapping area between each object to be matched by the reference object to be matched and the standard detection frame.

[0127] In this step, for any reference object to be matched, the image processing device can determine the overlapping area between each object to be matched that matches the reference object to be matched and the standard detection frame corresponding to the reference object to be matched, and determine the object to be matched whose overlapping area with the standard detection frame is greater than or equal to a predetermined threshold or greater than the predetermined threshold as the first-level object matching the reference object to be matched.

[0128] Optionally, in order to reduce the error caused by the absolute value of the overlapping area between the to-be-matched object and the standard detection frame in determining the correspondence between the to-be-matched object and the reference object, for any to-be-matched object that matches the to-be-matched reference object, the image processing device may convert the absolute value into a relative value. That is, based on the ratio of the overlapping area between the to-be-matched object and the standard detection frame corresponding to the to-be-matched reference object in the to-be-matched image and the area of ​​the to-be-matched object, the primary object that matches the to-be-matched reference object in the to-be-matched image is determined. Specifically, the image processing device may respectively determine the to-be-matched reference object whose area ratio with the standard detection frame corresponding to each to-be-matched reference object satisfies the second threshold condition as the primary object that matches the to-be-matched reference object.

[0129] In this optional implementation, the second threshold condition can be that the ratio of the overlapping areas of the standard detection frames corresponding to the reference objects to be matched with the object to be matched is greater than or equal to the second threshold, less than or equal to the second threshold, or falls within a second value range, etc., wherein the second threshold or the second value range can be determined statistically. Specifically, the second threshold or the second value range can be determined based on the ratio of the overlapping areas of the standard detection frames of multiple reference objects and the target objects corresponding to each reference object to the area of ​​the target objects corresponding to each reference object, wherein each reference object belongs to the same category and each target object belongs to the same category.

[0130] For example, when the category of the reference object is a motor vehicle and the category of the target object is a motor vehicle sign, the area of ​​the standard detection frame corresponding to motor vehicles such as passenger cars (including micro passenger cars, ordinary passenger cars, mid-range passenger cars, mid-to-high-end passenger cars and high-end passenger cars, etc.), buses, trucks, off-road vehicles, dump trucks, tractors, special-purpose vehicles, etc. can be obtained, and the area of ​​the sign detection frame of the motor vehicle sign of each motor vehicle can be obtained respectively, and the overlapping area of ​​the standard detection frame corresponding to each motor vehicle and the sign detection frame of the corresponding motor vehicle sign to the area ratio of the sign detection frame of the motor vehicle sign can be obtained, so that the minimum or maximum value of the area ratio can be determined as the second threshold, or the minimum value of the area ratio can be determined as the lower limit of the second value range, and the maximum value of the area ratio can be determined as the upper limit of the second value range.

[0131] In this optional implementation, for each reference object to be matched, the image processing device can determine, in each image to be detected, the object to be matched that is located within the reference object detection frame of the reference object to be matched as a candidate object corresponding to the reference object to be matched, and determine the standard detection frame corresponding to the reference object to be matched from the object detection frame of each candidate object based on the confidence of the object detection frame of each candidate object. Specifically, the image processing device can determine the object detection frame of the candidate object ranked first with the highest confidence as the standard detection frame corresponding to the reference object to be matched. The confidence of the detection frame is obtained when target detection is performed on the target image and the object to be detected is detected.

[0132] For example, in timestamp order, the target images of the target video, i.e., images 5-10, include reference object A. The image processing apparatus can determine candidate objects located within the reference object detection frame of reference object A in images 5, 6, 7, 8, 9, and 10, respectively. The candidate objects located within the reference object detection frame of reference object A in image 5 include objects B and C, the candidate objects located within the reference object detection frame of reference object A in image 6 include objects D, E, and F, the candidate objects located within the reference object detection frame of reference object A in image 7 include object G, the candidate objects located within the reference object detection frame of reference object A in image 8 include objects H and I, the candidate objects located within the reference object detection frame of reference object A in image 9 include object J, and the candidate objects located within the reference object detection frame of reference object A in image 10 include object K. The confidence score of the object detection frame of object C ranks first among all candidate objects, so the image processing apparatus can determine the object detection frame of object C as the standard detection frame corresponding to reference object A.

[0133] After determining the standard detection frame corresponding to each reference object to be matched, the image processing device can determine that the image to be detected where the standard detection frame is located is the standard image, and determine the relative position of the standard detection frame and the reference object to be matched according to the position of the standard detection frame in the standard image and the position of the reference object to be matched in the standard image, thereby determining the position of the standard detection frame in each image to be detected according to the position of the reference object to be matched in each image to be detected and the relative position of the standard detection frame and the reference object to be matched.

[0134] Figure 6 FIG is a schematic diagram of the relative positions of the reference object and the standard detection frame according to an embodiment of the present invention. Figure 6As shown, the detection frame 62 is the standard detection frame of the motor vehicle V in the standard image. The upper left corner coordinates of the detection frame 61 of the motor vehicle V in the standard image are (x0, y0), the width is W, and the height is H. The upper left corner coordinates of the detection frame 62 in the standard image are (x1, y1), and the lower right corner coordinates are (x2, y2). The image processing device can determine the coordinate axis with the upper left corner coordinate of the detection frame 61 of the motor vehicle V in the standard image as (x0, y0) as the origin, the coordinate axis with the upper left corner coordinate of the detection frame 61 as the starting point and the side in the horizontal direction as the positive direction of the x-axis, and the coordinate axis with the upper left corner coordinate of the detection frame 61 as the starting point and the side in the vertical direction as the positive direction of the y-axis, and determine the relative position coordinates of the upper left corner of the detection frame 62 relative to the motor vehicle V as (x1-x0, y1-y0) and the relative position coordinates of the lower right corner as (x2-x0, y2-y0) based on the coordinate axis, thereby determining (x1-x0, y1-y0) and (x2-x0, y2-y0) as the relative positions of the detection frame 62 and the motor vehicle A.

[0135] Figure 7 FIG. 1 is another schematic diagram of the relative positions of the reference object and the standard detection frame according to an embodiment of the present invention. Figure 6 ,like Figure 7 As shown, the detection frame 62' is the standard detection frame of the motor vehicle V in the image to be detected. The coordinates of the upper left corner of the detection frame 71 of the motor vehicle V in the image to be detected are (x0', y0'), the width is W', and the height is H'. The image processing device can determine the coordinate axis origin with the coordinates of the upper left corner of the detection frame 71 of the motor vehicle V in the standard image being (x0', y0'), the coordinate axis with the coordinates of the upper left corner of the detection frame 71 as the starting point and the horizontal side as the positive direction of the x-axis, and the coordinate axis with the coordinates of the upper left corner of the detection frame 71 as the starting point and the vertical side as the positive direction of the y-axis, and determine the relative position coordinates of the detection frame 62' relative to the upper left corner of the motor vehicle V in the image to be detected as (W' / W(x1-x0),H' / H(y1-y0)). , the relative position coordinates of the lower right corner in the image to be detected are (W' / W(x2-x0), H' / H(y2-y0)), thereby determining that the horizontal coordinate x1'=x0'+W' / W(x1-x0) and the vertical coordinate y1'=y0'+H' / H(y1-y0) of the upper left corner coordinate of the detection frame 62' in the image to be detected are; the horizontal coordinate x2'=x0'+W' / W(x2-x0) and the vertical coordinate y2'=y0'+H' / H(y2-y0) of the upper left corner coordinate in the image to be detected are.

[0136] Still Figure 4As shown in the example, detection frame 48 is the standard detection frame corresponding to vehicle A in image 40. The image processing device can determine the ratio of the overlapping area between detection frame 48 and detection frame 44 of vehicle mark a to the area of ​​detection frame 44 of vehicle mark a, and the ratio of the overlapping area between detection frame 48 and detection frame 45 of vehicle mark b to the area of ​​detection frame 45 of vehicle mark b. Since the area ratio corresponding to vehicle mark a satisfies the second threshold condition, the image processing device can determine that the primary object corresponding to vehicle A is vehicle mark a. At this point, vehicle A and vehicle mark a are matched one-to-one, so the image processing device can directly determine the correspondence between vehicle A and vehicle mark a.

[0137] Step S370 : determining the target object to be matched by the reference object to be matched in the image to be detected according to the confidence level of the object detection frame of each primary object in the image to be detected.

[0138] After determining the first-level objects corresponding to each reference object to be matched, for any reference object to be matched, the image processing device can respectively determine the confidence of the object detection frame of each first-level object matched by the reference object to be matched, and determine the first-level object corresponding to the object detection frame with the highest confidence as the target object matched by the reference object to be matched in the image to be detected.

[0139] It is easy to understand that in this optional implementation, if there is a one-to-one matching relationship between at least one reference object to be matched and the first-level object, the image processing device can directly determine the corresponding relationship between the at least one reference object to be matched and the first-level object, and there is no need to execute steps S370 and S380.

[0140] Step S380: determining a corresponding relationship between each reference object to be matched and the matching target object.

[0141] After the above steps, each to-be-matched object is matched one-to-one with each reference object, so the image processing device can determine the corresponding relationship between each to-be-matched reference object and the matching target object.

[0142] Step S400 : determining the object identifier of each to-be-matched object in each target image according to the reference object identifier and the corresponding relationship of each reference object, so as to track the to-be-matched object according to its position in each target image.

[0143] After determining the correspondence between each reference object and each object to be matched in at least one target image, the image processing device can determine the object identification of each object to be matched in each target image based on the reference object identification of each reference object, and mark the object detection frame of each object to be matched in each target image based on the object identification of each object to be matched, thereby achieving target tracking of the object to be matched with lower computing resource consumption and higher target tracking efficiency.

[0144] Still Figure 4 As shown in the example, when the image processing device determines that the correspondence between each reference object and the object to be matched is one-to-one correspondence between motor vehicle A and motor vehicle mark a, one-to-one correspondence between motor vehicle B and motor vehicle mark c, and one-to-one correspondence between motor vehicle C and motor vehicle mark d, the server can determine that motor vehicle mark a is the motor vehicle mark of motor vehicle A, motor vehicle mark c is the motor vehicle mark of motor vehicle B, and motor vehicle mark d is the motor vehicle mark of motor vehicle C, and determine that detection frame 44 is the detection frame of the motor vehicle mark of motor vehicle A, determine that detection frame 46 is the detection frame of the motor vehicle mark of motor vehicle B, and determine that detection frame 47 is the detection frame of the motor vehicle mark of motor vehicle C. That is, the position of the motor vehicle mark of motor vehicle A in image 40, the position of the motor vehicle mark of motor vehicle B in image 40, and the position of the motor vehicle mark of motor vehicle C in image 40 are determined, thereby realizing target tracking of the motor vehicle mark of motor vehicle A, the motor vehicle mark of motor vehicle B, and the motor vehicle mark of motor vehicle C.

[0145] After acquiring the video, the embodiment of the present invention performs target detection on each image in the video, determines the object to be matched in each image, and determines the correspondence between each object to be matched and each reference object in the image based on the overlapping area of ​​each object to be matched and each reference object in the same image, and then determines the object identification of each object to be matched in each image based on the reference object identification of the reference object and the correspondence between each object to be matched and each reference object in the same image, so as to track the object to be matched according to its position in each image. In the embodiment of the present invention, by performing target detection on the image to be detected and matching the reference object to determine the object identification of each object to be matched in each image, it is possible to reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking.

[0146] Figure 8 FIG. 1 is a flow chart of a target tracking method according to a second embodiment of the present invention. Figure 8 As shown, the method of this embodiment includes the following steps:

[0147] Step S100 ′: obtaining a target video.

[0148] In this embodiment, the implementation of step S100 ′ is similar to that of step S100 , and will not be described in detail herein.

[0149] Step S200 ′: performing target detection on at least one target image to determine an object to be matched in each target image.

[0150] In this embodiment, the implementation of step S200 ′ is similar to that of step S200 , and will not be described in detail herein.

[0151] Step S300 ′: determining the correspondence between each to-be-matched object and each reference object in the same target image according to the first overlapping area between each to-be-matched object and each reference object in the same target image.

[0152] In this embodiment, the implementation of step S300 ′ is similar to that of step S300 , and will not be described in detail herein.

[0153] Step S400 ′: determining the object identifier of each to-be-matched object in each target image according to the reference object identifier and the corresponding relationship of each reference object, so as to track the to-be-matched object according to its position in each target image.

[0154] In this embodiment, the implementation of step S400 ′ is similar to that of step S400 , and will not be described in detail herein.

[0155] Step S500 ′: in response to at least one reference object in the target image but no corresponding object to be matched is detected, the target image in which no corresponding object to be matched is detected is determined as the current image.

[0156] In this embodiment, if there is at least one reference object in any target image whose corresponding object to be matched is not detected, and the objects to be matched corresponding to the reference object are detected in multiple target images in the target video, it means that the object to be matched corresponding to the reference object actually exists but has not been recognized by the target detection algorithm. Therefore, the image processing device can determine the target image as the current image.

[0157] In this step, the image processing device may iteratively determine each target image in which the corresponding object to be matched is not detected as the current image, or may simultaneously determine all target images in which the corresponding object to be matched is not detected as the current image.

[0158] Step S600 ′: determining a target position of the object to be matched according to at least one of the first position of the object to be matched and the second position of the object to be matched.

[0159] In this embodiment, the first position of the object to be matched is the position of the object to be matched as detected in the target image preceding the current image, the second position of the object to be matched is the position of the object to be matched as detected in the target image following the current image, and the target position of the object to be matched is the position of the object to be matched in the current image. In this step, the above positions can be the absolute position of the object to be matched in the target image or the relative position of the object to be matched with respect to a reference object. Specifically, the position of the object to be matched can be represented by the coordinates of the upper left corner and the lower right corner of the object detection frame of the object to be matched.

[0160] For example, the target video contains a sequence of 120 target images, arranged in timestamp order as images 1 through 120. In image 30, there is a motor vehicle (i.e., a reference object) for which no corresponding motor vehicle sign (i.e., an object to be matched) is detected. However, the corresponding motor vehicle sign is detected in images 20 through 29 and images 31 through 120. Therefore, the image processing device can determine image 30 as the current image and determine the relative position of the motor vehicle sign relative to the motor vehicle in image 30 based on the relative position of the motor vehicle sign detected in at least some of the target images in images 20 through 29 and images 31 through 120. Thus, the position of the motor vehicle sign in image 30 can be determined based on the relative position of the motor vehicle sign in image 30 and the position of the motor vehicle.

[0161] In an optional implementation, the image processing device may perform linear fitting based on at least one of a plurality of continuous first positions and a plurality of continuous second positions to obtain a position fitting function of the object to be matched, and determine the target position of the object to be matched based on the position fitting function and the timestamp of the current image. The at least one of the plurality of continuous first positions and a plurality of continuous second positions referred to herein means that the image processing device may determine the position fitting function based on the plurality of continuous first positions, may determine the position fitting function based on the plurality of continuous second positions, or may determine the position fitting function based on the plurality of continuous first positions and a plurality of continuous second positions.

[0162] Furthermore, when the current image is the first target image in the target video, that is, the first frame target image, the image processing device can perform backward (that is, reverse) Kalman filtering based on multiple consecutive second positions, that is, perform Kalman filtering in reverse order of timestamp order, obtain a position fitting function of the object to be matched, and thus determine the position of the object to be matched in the current image based on the timestamp of the current image. Optionally, this embodiment can also implement linear fitting through other methods, such as the least squares method, which is not limited in this embodiment.

[0163] Figure 9 FIG is a schematic diagram of determining the position of the object to be matched in the current image according to the position of the object to be matched corresponding to the reference object in the target image according to an embodiment of the present invention. Figure 9As shown, the target video contains a sequence of 100 target images, arranged in timestamp order from image P1 to image P100. Line segment L1 indicates that vehicle 7A (the reference object) was tracked in all images P1 through P100. Line segment set L2, consisting of multiple line segments, indicates that vehicle mark 7a of vehicle 7A was not detected in image P1, but was continuously detected in images P2 through P30. Therefore, when the position 91 is the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can perform Kalman filtering on the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a in image P2-image P30 in the order of image P30 to image P2 to obtain a linear fitting function of the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a, thereby determining the position 91 of the motor vehicle mark 7a; when the position 91 is the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can perform Kalman filtering on the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a in image P2-image P30 in the order of image P30 to image P2 to obtain a linear fitting function of the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a, thereby determining the position 91 of the motor vehicle mark 7a.

[0164] When the current image is the last target image in the target video, that is, the last frame target image, the image processing device can perform forward (that is, positive) Kalman filtering based on multiple consecutive first positions, that is, perform Kalman filtering in positive order according to the timestamp order, and obtain the position fitting function of the object to be matched, so as to determine the position of the object to be matched in the current image according to the timestamp of the current image.

[0165] Still Figure 9 As shown in the example, line segment set L2 also indicates that the vehicle sign 7a was not detected in image P100, but was continuously detected in images P80-P99. Therefore, when position 94 is the coordinate of the upper left corner of the sign detection frame of the vehicle sign 7a, the image processing device can perform Kalman filtering on the coordinates of the upper left corner of the sign detection frame of the vehicle sign 7a in images P80-P99 in the order from image P80 to image P99 to obtain a linear fitting function of the coordinates of the upper left corner of the sign detection frame of the vehicle sign 7a, thereby determining position 94 of the vehicle sign 7a. When position 94 is the coordinate of the lower right corner of the sign detection frame of the vehicle sign 7a, the image processing device can perform Kalman filtering on the coordinates of the lower right corner of the sign detection frame of the vehicle sign 7a in images P80-P99 in the order from image P80 to image P99 to obtain a linear fitting function of the coordinates of the lower right corner of the sign detection frame of the vehicle sign 7a, thereby determining position 94 of the vehicle sign 7a.

[0166] Optionally, when the current image is a target image at a position other than the first and last positions in the target video, that is, a target image at a middle position, the image processing device can perform bidirectional Kalman filtering based on multiple consecutive first positions and multiple consecutive second positions to obtain a position fitting function of the object to be matched, thereby determining the position of the object to be matched in the current image according to the timestamp of the current image.

[0167] Still Figure 9 As shown in the example, the line segment set L2 also indicates that the motor vehicle sign 7a is not detected in the image P31, and the motor vehicle sign 7a is continuously detected in the images P2 to P30 and the images P32 to P78. Therefore, when the position 92 is the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can perform bidirectional Kalman filtering on the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a in image P2-image P30 and image P32-image P78 in the order of image P2 to image P30 and from image P32 to image P78 to obtain a linear fitting function of the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a, thereby determining the position 92 of the motor vehicle mark 7a; when the position 92 is the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can perform bidirectional Kalman filtering on the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a in image P2-image P30 and image P32-image P78 in the order of image P2 to image P30 and from image P32 to image P78 to obtain a linear fitting function of the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a, thereby determining the position 92 of the motor vehicle mark 7a.

[0168] Optionally, when the current image is a target image at a position other than the first and last positions in the target video, the image processing device can determine from each first position the position of the object to be matched detected in the target image closest to the timestamp of the current image as the first target position, and determine from each second position the position of the object to be matched detected in the target image closest to the timestamp of the current image as the second target position, thereby interpolating the target position according to the first target position and the second target position to determine the position of the object to be matched in the current image.

[0169] Specifically, when the first target position is the relative position of the object to be matched and the reference object detected in the target image closest to the timestamp of the current image and with a timestamp before the current image, and the second target position is the relative position of the object to be matched and the reference object detected in the target image closest to the timestamp of the current image and with a timestamp after the current image, taking the coordinates (x, y) of the relative position of the upper left corner of the object to be matched and the reference object in the current image as an example, the horizontal coordinate of the relative position of the upper left corner of the object to be matched and the reference object is The vertical coordinate of the relative position of the upper left corner Where x0 is the horizontal coordinate of the relative position of the upper left corner of the first target position, x1 is the horizontal coordinate of the relative position of the upper left corner of the second target position, t is the timestamp of the current image, t0 is the timestamp corresponding to the first target position, t1 is the timestamp corresponding to the second target position, y0 is the vertical coordinate of the relative position of the upper left corner of the first target position, and y1 is the vertical coordinate of the relative position of the upper left corner of the second target position.

[0170] Still Figure 9 As shown in the example, the line segment set L2 also indicates that the motor vehicle sign 7a is not detected in the image P79, and the motor vehicle sign 7a is continuously detected in the images P32 to P78 and the images P80 to P99. Therefore, the image processing device can determine that image P30 and image P32 are the target images closest to the timestamp of image P31, and when position 93 is the upper left corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can interpolate according to the upper left corner relative position coordinates of the motor vehicle mark 7a and the motor vehicle 7A in image P30 and image P32 to obtain the upper left corner relative position coordinates of the motor vehicle mark 7a and the motor vehicle 7A in image P31, thereby determining the upper left corner relative position 93 of the motor vehicle mark 7a in image P31; when position 93 is the lower right corner coordinate of the mark detection frame of the motor vehicle mark 7a, the image processing device can interpolate according to the lower right corner relative position coordinates of the motor vehicle mark 7a and the motor vehicle 7A in image P30 and image P32 to obtain the lower right corner relative position coordinates of the motor vehicle mark 7a and the motor vehicle 7A in image P31, thereby determining the lower right corner relative position 93 of the motor vehicle mark 7a in image P31.

[0171] In this way, when the object to be matched corresponding to the reference object cannot be detected in the target image, the position of the object to be matched in the target image can be determined according to the relative positions of the reference object and the object to be matched in other target images, thereby achieving tracking of the object to be matched.

[0172] After acquiring the video, the embodiment of the present invention performs target detection on each image in the video, determines the object to be matched in each image, and determines the correspondence between each object to be matched and each reference object in the image based on the overlapping area of ​​each object to be matched and each reference object in the same image, and then determines the object identification of each object to be matched in each image based on the reference object identification of the reference object and the correspondence between each object to be matched and each reference object in the same image, so as to track the object to be matched according to its position in each image. In the embodiment of the present invention, by performing target detection on the image to be detected and matching the reference object to determine the object identification of each object to be matched in each image, it is possible to reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking.

[0173] Figure 10 FIG is a schematic diagram of a target tracking device according to a third embodiment of the present invention. Figure 10 As shown, the apparatus of this embodiment includes a video acquisition unit 1001 , a target detection unit 1002 , a relationship determination unit 1003 and an identification determination unit 1004 .

[0174] Among them, the video acquisition unit 1001 is used to acquire the target video, which is a sequence composed of multiple target images, each of which includes a reference object; the target detection unit 1002 is used to perform target detection on at least one of the target images to determine the objects to be matched in each of the target images; the relationship determination unit 1003 is used to determine the correspondence between each of the objects to be matched and each of the reference objects in the same target image based on the first overlapping area of ​​each of the objects to be matched and each of the reference objects in the same target image; the identification determination unit 1004 is used to determine the object identification of each of the objects to be matched in each of the target images based on the reference object identification of each of the reference objects and the corresponding relationship, so as to track the objects to be matched according to the positions of the objects to be matched in each of the target images.

[0175] Furthermore, the relationship determination unit 1003 includes an image determination subunit, a primary determination subunit, a secondary determination subunit and a target determination subunit.

[0176] Among them, the image determination subunit is used to determine each of the target images as the image to be detected; the first-level determination subunit is used to determine the first-level reference object matched by each of the objects to be matched in the image to be detected according to the first overlapping area of ​​each of the objects to be matched and each of the reference objects in the image to be detected; the second-level determination subunit is used to determine the second-level reference object matched by each of the objects to be matched in the image to be detected according to the first area ratio of each of the objects to be matched and each of the matched first-level reference objects; the target determination subunit is used to determine the target reference object matched by each of the objects to be matched in the image to be detected according to the distance between each of the second-level reference objects matched by each of the objects to be matched and the image acquisition device, so as to determine the corresponding relationship according to the matching relationship between each of the objects to be matched and each of the reference objects.

[0177] Furthermore, the first-level determination subunit is further configured to determine, for each of the objects to be matched, the first-level reference object to be matched in the image to be detected according to a ratio between the corresponding first overlapping areas and the second area of ​​the object to be matched.

[0178] Furthermore, the target determination subunit includes a first reference object determination module, a first-level determination module, a target determination module and a first relationship determination module.

[0179] Among them, the first reference object determination module is used to determine each of the reference objects that matches the multiple objects to be matched as a reference object to be matched in response to the matching relationship representation that there is at least one reference object that matches the multiple objects to be matched; the first-level determination module is used to determine the first-level object that matches the reference object to be matched in the image to be detected based on the second overlapping area of ​​each object to be matched matched by the reference object to be matched and the standard detection frame, and the standard detection frame is the object detection frame of the object to be matched whose relative position with the reference object to be matched remains unchanged; the target determination module is used to determine the target object that the reference object to be matched matches in the image to be detected based on the confidence of the object detection frame of each first-level object in the image to be detected; the first relationship determination module is used to determine the corresponding relationship based on each reference object to be matched and the matched target object.

[0180] Furthermore, the first-level determination module is further used to determine the first-level object that matches the reference object to be matched in the image to be detected based on the ratio of the second overlapping area between the same object to be matched that matches the reference object to be matched and the standard detection frame to the third area of ​​the same object to be matched.

[0181] Furthermore, the standard detection frame is determined by an object determination module and a detection frame determination module.

[0182] Among them, the object determination module is used to determine the candidate objects in each of the images to be detected, and the candidate objects are the objects to be matched that are located within the reference object detection frame of the reference object to be matched; the detection frame determination module is used to determine the standard detection frame from each of the object detection frames based on the confidence of the object detection frame of each of the candidate objects.

[0183] Furthermore, the position of the standard detection frame is determined by an image determination module, a relative position determination module and a position determination module.

[0184] Among them, the image determination module is used to determine the image to be detected where the standard detection frame is located as the standard image; the relative position determination module is used to determine the relative position of the standard detection frame and the reference object to be matched according to the position of the standard detection frame in the standard image and the position of the reference object to be matched in the standard image; the position determination module is used to determine the position of the standard detection frame in each of the images to be detected according to the position of the reference object to be matched in each of the images to be detected and the relative position.

[0185] Furthermore, the target determination subunit further includes a second reference object determination module and a second relationship determination module.

[0186] Among them, the second reference object determination module is used to determine the reference object that matches one of the objects to be matched as a matched reference object in response to the matching relationship representation that there is at least one reference object that matches one of the objects to be matched; the second relationship determination module is used to determine the corresponding relationship based on the matched reference object and the matched object to be matched.

[0187] Furthermore, the device further includes an image determination unit and a position determination unit.

[0188] Among them, the image determination unit is used to determine the target image in which the corresponding object to be matched is not detected as the current image in response to the presence of at least one reference object in the target image; the position determination unit is used to determine the target position of the object to be matched based on at least one of the first position of the object to be matched and the second position of the object to be matched, the first position being the position of the object to be matched detected in the target image sequentially before the current image, the second position being the position of the object to be matched detected in the target image sequentially after the current image, and the target position being the position of the object to be matched in the current image.

[0189] Furthermore, the position determination unit includes a function determination subunit and a first position determination subunit.

[0190] Among them, the function determination subunit is used to perform linear fitting based on multiple consecutive first positions and at least one of multiple consecutive second positions to obtain the position fitting function of the object to be matched; the first position determination subunit is used to determine the target position according to the position fitting function and the timestamp of the current image.

[0191] Furthermore, the function determination subunit includes a first function determination module or a second function determination module.

[0192] Among them, the first function determination module is used to perform forward Kalman filtering based on multiple continuous first positions to obtain the position fitting function; the second function determination module is used to perform backward Kalman filtering based on multiple continuous second positions to obtain the position fitting function.

[0193] Furthermore, the position determining unit includes a second position determining subunit, a third position determining subunit and a fourth position determining subunit.

[0194] Among them, the second position determination subunit is used to determine the first target position from each first position, and the first target position is the first position of the object to be matched detected in the target image closest to the timestamp of the current image; the third position determination subunit is used to determine the second target position from each second position, and the second target position is the second position of the object to be matched detected in the target image closest to the timestamp of the current image; the fourth position determination subunit is used to interpolate the target position based on the first target position and the second target position to determine the target position.

[0195] After acquiring the video, the embodiment of the present invention performs target detection on each image in the video, determines the object to be matched in each image, and determines the correspondence between each object to be matched and each reference object in the image based on the overlapping area of ​​each object to be matched and each reference object in the same image, and then determines the object identification of each object to be matched in each image based on the reference object identification of the reference object and the correspondence between each object to be matched and each reference object in the same image, so as to track the object to be matched according to its position in each image. In the embodiment of the present invention, by performing target detection on the image to be detected and matching the reference object to determine the object identification of each object to be matched in each image, it is possible to reduce the consumption of computing resources while achieving target tracking and improve the efficiency of target tracking.

[0196] Figure 11 FIG is a schematic diagram of an electronic device according to a fourth embodiment of the present invention. Figure 11 As shown, electronic device 110 is a general-purpose data processing device that includes a general-purpose computer hardware structure, including at least a processor 1101 and a memory 1102. Processor 1101 and memory 1102 are connected via a bus 1103. Memory 1102 is suitable for storing instructions or programs executable by processor 1101. Processor 1101 can be an independent microprocessor or a collection of one or more microprocessors. Thus, processor 1101 executes the instructions stored in memory 1102 to perform the method flow of the embodiment of the present invention described above to process data and control other devices. Bus 1103 connects the above-mentioned multiple components together and connects the above-mentioned components to a display controller 1104 and a display device as well as an input / output (I / O) device 1105. Input / output (I / O) device 1105 can be a mouse, keyboard, modem, network interface, touch input device, somatosensory input device, printer, and other devices known in the art. Typically, input / output devices 1105 are connected to the system through input / output (I / O) controllers 1106 .

[0197] It will be understood by those skilled in the art that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] The present application is described with reference to flowcharts of methods, apparatuses (devices), and computer program products according to embodiments of the present application. It should be understood that each process in the flowcharts can be implemented by computer program instructions.

[0199] These computer program instructions may be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0200] These computer program instructions can also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0201] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute part or all of the above method embodiments.

[0202] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by specifying relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0203] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A target tracking method, characterized in that: The method comprises: Acquire a target video, where the target video is a sequence consisting of a plurality of target images, each of which includes a reference object; Performing target detection on at least one of the target images to determine an object to be matched in each of the target images; determining a correspondence between each of the objects to be matched and each of the reference objects in the same target image according to a first overlapping area between each of the objects to be matched and each of the reference objects in the same target image; The object identifier of each of the reference objects is determined in each of the target images according to the reference object identifier and the corresponding relationship, so as to track the object to be matched according to the position of the object to be matched in each of the target images.

2. The method according to claim 1, characterized in that Determining the correspondence between each of the to-be-matched objects and each of the reference objects in the same target image according to the first overlapping area of ​​each of the to-be-matched objects and each of the reference objects in the same target image includes: Determine each of the target images as an image to be detected; Determining, in the image to be detected, a primary reference object to be matched with each of the objects to be matched, according to the first overlapping area between each of the objects to be matched and each of the reference objects in the image to be detected; Determining, in the image to be detected, a secondary reference object to be matched by each of the objects to be matched according to a first area ratio between each of the objects to be matched and each of the primary reference objects to be matched; The target reference object matched by each to-be-matched object in the to-be-matched image is determined according to the distance between each to-be-matched object and the image acquisition device, so as to determine the corresponding relationship according to the matching relationship between each to-be-matched object and each reference object.

3. The method according to claim 2, characterized in that Determining, in the image to be detected, a primary reference object to which each of the objects to be matched matches based on the first overlapping area between each of the objects to be matched and each of the reference objects in the image to be detected includes: For each of the objects to be matched, the matching primary reference object is determined in the image to be detected according to the ratio of the corresponding first overlapping area to the second area of ​​the object to be matched.

4. The method according to claim 2, characterized in that Determining the corresponding relationship according to the matching relationship between each of the to-be-matched objects and each of the reference objects includes: In response to the matching relationship representation that at least one of the reference objects matches a plurality of the objects to be matched, determining each of the reference objects that matches the plurality of objects to be matched as a reference object to be matched; determining, in the image to be detected, a primary object matched by the reference object to be matched based on a second overlapping area between each of the objects to be matched and a standard detection frame, wherein the standard detection frame is an object detection frame of the object to be matched whose relative position to the reference object to be matched remains unchanged; determining a target object to be matched by the reference object to be matched in the image to be detected according to the confidence of the object detection frame of each of the first-level objects in the image to be detected; The corresponding relationship is determined according to each of the reference objects to be matched and the target object to be matched.

5. The method according to claim 4, characterized in that The step of determining the primary object matched by the reference object to be matched in the image to be detected according to the second overlapping area between each of the objects to be matched and the standard detection frame includes: The primary object matching the reference object to be matched is determined in the image to be detected according to the ratio of the second overlapping area between the same object to be matched that matches the reference object to be matched and the standard detection frame to a third area of ​​the same object to be matched.

6. The method according to claim 4, characterized in that The standard detection frame is determined as follows: Determine a candidate object in each of the images to be detected, wherein the candidate object is the object to be matched that is located within a reference object detection frame of the reference object to be matched; The standard detection frame is determined from the object detection frames according to the confidence of the object detection frame of each candidate object.

7. The method according to claim 6, characterized in that The position of the standard detection frame is determined as follows: Determining the image to be detected where the standard detection frame is located as a standard image; Determining the relative position of the standard detection frame and the reference object to be matched according to the position of the standard detection frame in the standard image and the position of the reference object to be matched in the standard image; The position of the standard detection frame in each of the images to be detected is determined according to the position of the reference object to be matched in each of the images to be detected and the relative position.

8. The method according to claim 2 or 4, characterized in that Determining the corresponding relationship according to the matching relationship between each of the to-be-matched objects and each of the reference objects includes: In response to the matching relationship representation that at least one of the reference objects matches one of the objects to be matched, determining the reference object that matches one of the objects to be matched as a matched reference object; The corresponding relationship is determined according to the matched reference object and the matched object to be matched.

9. The method according to claim 1, characterized in that The method further comprises: In response to at least one of the reference objects in the target image not detecting the corresponding object to be matched, determining the target image not detecting the corresponding object to be matched as the current image; The target position of the object to be matched is determined based on at least one of a first position of the object to be matched and a second position of the object to be matched, wherein the first position is the position of the object to be matched detected in the target image sequentially before the current image, the second position is the position of the object to be matched detected in the target image sequentially after the current image, and the target position is the position of the object to be matched in the current image.

10. The method according to claim 9, characterized in that Determining the target position of the object to be matched according to at least one of the first position of the object to be matched and the second position of the object to be matched includes: Performing linear fitting based on at least one of a plurality of continuous first positions and a plurality of continuous second positions to obtain a position fitting function of the object to be matched; The target position is determined according to the position fitting function and a timestamp of the current image.

11. The method according to claim 10, characterized in that The performing linear fitting based on at least one of the plurality of continuous first positions and the plurality of continuous second positions to obtain the position fitting function of the object to be matched includes: Performing forward Kalman filtering based on a plurality of consecutive first positions to obtain the position fitting function; or Backward Kalman filtering is performed based on a plurality of continuous second positions to obtain the position fitting function.

12. The method according to claim 9, characterized in that Determining the target position of the object to be matched according to at least one of the first position of the object to be matched and the second position of the object to be matched includes: Determine a first target position from each of the first positions, where the first target position is the first position of the object to be matched detected in the target image with a timestamp closest to that of the current image; Determine a second target position from each of the second positions, where the second target position is the second position of the object to be matched detected in the target image with a timestamp closest to that of the current image; The target position is determined by interpolating the target position based on the first target position and the second target position.

13. A target tracking device, characterized in that: The device comprises: a video acquisition unit, configured to acquire a target video, wherein the target video is a sequence composed of a plurality of target images, each of which includes a reference object; a target detection unit, configured to perform target detection on at least one of the target images and determine an object to be matched in each of the target images; a relationship determining unit, configured to determine a corresponding relationship between each of the to-be-matched objects and each of the reference objects in the same target image according to a first overlapping area between each of the to-be-matched objects and each of the reference objects in the same target image; The identification determination unit is used to determine the object identification of each of the objects to be matched in each of the target images according to the reference object identification of each of the reference objects and the corresponding relationship, so as to track the objects to be matched according to their positions in each of the target images.

14. An electronic device comprising a memory and a processor, characterized in that: The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images