Target tracking method, target detection method and device

Abstract

The present application discloses a target tracking method, a target detection method and a device, which relate to the field of artificial intelligence (AI), and are used to solve the problem of unstable display of tracking frames in multi-target tracking, and are related to computer vision, target detection and tracking, etc. The target tracking method includes: performing target detection on a frame of target image to obtain at least one detection frame, and performing tracking detection on the target image to obtain at least one tracking frame; determining the stability level of at least one detection frame according to a historical detection frame and a historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on other one or more frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on other one or more frames of images; matching the first detection frame with the at least one tracking frame respectively, and updating the target tracking frame displayed in the target image according to the matching result, wherein the first detection frame does not include a detection frame with a stability level of the first level in at least one detection frame.

Description

Technical Field

[0001] The present application relates to the field of terminal artificial intelligence (AI), and in particular to a target tracking method, a target detection method and a device. Background Art

[0002] Object tracking is an important technology in computer vision. Object tracking can include single object tracking and multi-object tracking. Single object tracking is used to track the motion trajectory of a single object in the video screen, and multi-object tracking is used to track the motion trajectory of multiple objects in the video screen at the same time.

[0003] During target tracking, a tracking frame is displayed around the target to identify the target. Existing multi-target tracking technology has unstable problems such as jitter and intermittent display of the tracking frame due to user hand shaking or uncertain target movement speed. For end-side application scenarios such as mobile phones and virtual reality devices, this instability is unacceptable. Summary of the invention

[0004] Embodiments of the present application provide a target tracking method, a target detection method and a device for solving the problem of unstable display of tracking frames in multi-target tracking.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a target tracking method is provided, comprising: performing target detection on a frame of a target image to obtain at least one detection frame, and performing tracking detection on the target image to obtain at least one tracking frame; determining a stability level of the at least one detection frame based on a historical detection frame and a historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on one or more other frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames of images; matching a first detection frame with the at least one tracking frame respectively, and updating the target tracking frame displayed in the target image based on the matching result, wherein the first detection frame does not include a detection frame whose stability level is the first level among the at least one detection frame.

[0007] The target tracking method provided by the embodiment of the present application performs target detection on a frame of target image to obtain at least one detection frame, and performs tracking detection on the target image to obtain at least one tracking frame. According to the historical detection frame and the historical tracking frame, the stability level of the at least one detection frame is determined, wherein the historical detection frame is a detection frame obtained by performing target detection on other one or more frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on other one or more frames of images. The first detection frame is matched with the at least one tracking frame respectively, and the target tracking frame displayed in the image is updated according to the matching result, wherein the first detection frame does not include the detection frame with a stability level of the first level in the at least one detection frame. By updating the tracking frame without including the detection frame with high stability of the first level, the display of the tracking frame is made more stable, which can solve the problem of unstable display of the tracking frame in multi-target tracking.

[0008] In a possible implementation, the first level is low stability, that is, the tracking frame is updated by a detection frame that is not low stability, so that the display of the tracking frame is more stable.

[0009] In a possible implementation, the stability level of the at least one detection frame is determined based on the historical detection frames and the historical tracking frames, including: removing the detection frames that meet the first preset condition from the at least one detection frame to obtain the remaining detection frames; matching the remaining detection frames with the historical detection frames to obtain the stability levels of the remaining detection frames, the remaining detection frames including the second detection frames with low stability (e.g., based on the matching results, a stability level is determined for each of the remaining detection frames, wherein a portion of the detection frames are determined to have low stability, which can be referred to as the second detection frames. In addition, in some possible scenarios, a portion of the detection frames can also be determined to have high stability. Furthermore, in some possible scenarios, a portion of the detection frames can also be determined to have medium stability); matching the second detection frame with the historical tracking frame to update the stability level of the second detection frame.

[0010] In a possible implementation, the detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a first threshold; a detection frame whose area ratio relative to the target image is lower than a second threshold; and a repeated detection frame for the same target. The remaining detection frames obtained in the above manner remove the detection frames with lower values, thereby reducing the workload of the subsequent matching process with the at least one tracking frame.

[0011] In a possible implementation, updating the tracking frame displayed in the target image according to the matching result includes: creating a new tracking frame in the target image according to the third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched. That is, for a detection frame with a higher stability level, if the tracking frame cannot be matched, it means that it is a new target, and thus a new tracking frame is created.

[0012] In a possible implementation manner, the second level is high stability.

[0013] In a possible implementation, updating the tracking frame displayed in the target image according to the matching result includes: suppressing or penalizing a first tracking frame displayed in the target image, wherein the first tracking frame is a tracking frame of at least one tracking frame whose matching result is unmatched or low matching degree.

[0014] In a possible implementation, the method further includes: adjusting the frequency of target detection according to the acceleration information. For example, the greater the acceleration of the image sensor, the faster the terminal device moves or the more severe the shaking. The target detection frequency and the tracking detection frequency can be increased to improve the detection accuracy. Otherwise, it means that the terminal device moves slowly or the shaking is not severe. The target detection frequency and the tracking detection frequency can be reduced to reduce power consumption.

[0015] In a second aspect, a target detection method is provided, including: training a deep neural network according to a foreground object training set and a key focus object training set to obtain a trained deep neural network, the foreground object training set including a first image set and position information of foreground objects in each image in the first image set, and the key focus object training set including the first image set and position information of key focus objects in each image in the first image set, and category information of key focus objects in each image in the first image set; inputting a target image into the trained deep neural network, and obtaining position information of a foreground frame, position information of a key category frame, and category information corresponding to the key category frame through calculation, wherein the foreground frame is a detection frame that identifies the foreground object, and the key category frame is a detection frame that identifies the key focus object.

[0016] The object detection method provided in the embodiment of the present application combines the category information corresponding to the key category box, decouples classification and recognition, can support more classifications, and its classification accuracy is relatively higher. Therefore, it can also be applied to many scenarios such as image search, celebrity search, friend search, text segmentation, car model recognition, banknote recognition, etc.

[0017] In a third aspect, a target tracking device is provided, including: an acquisition module, used to perform target detection on a frame of a target image to obtain at least one detection frame, and to perform tracking detection on the target image to obtain at least one tracking frame; a determination module, used to determine the stability level of the at least one detection frame based on a historical detection frame and a historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on one or more other frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames of images; an update module, used to match a first detection frame with the at least one tracking frame respectively, and update the target tracking frame displayed in the target image according to the matching result, wherein the first detection frame does not include a detection frame with a stability level of the first level in the at least one detection frame.

[0018] In a possible implementation, the first level is low stability.

[0019] In a possible implementation, the determination module is specifically used to: remove the detection frame that meets the first preset condition from the at least one detection frame to obtain the remaining detection frame; match the remaining detection frame with the historical detection frame to obtain the stability level of the remaining detection frame, the remaining detection frame includes a second detection frame with low stability; match the second detection frame with the historical tracking frame to update the stability level of the second detection frame.

[0020] In a possible implementation, the detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a first threshold; a detection frame whose area ratio relative to the target image is smaller than a second threshold; and a repeated detection frame for the same target.

[0021] In a possible implementation, the updating module is specifically configured to: create a new tracking frame in the target image according to a third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched.

[0022] In a possible implementation manner, the second level is high stability.

[0023] In a possible implementation, the updating module is specifically configured to suppress or punish a first tracking frame displayed in the target image, wherein the first tracking frame is a tracking frame whose matching result is unmatched or low matching degree among the at least one tracking frame.

[0024] In a possible implementation, the system further includes: an adjustment module, configured to adjust the frequency of target detection according to acceleration information.

[0025] In a fourth aspect, a target detection device is provided, including: a training module, used to train a deep neural network according to a foreground object training set and a key focus object training set to obtain a trained deep neural network, the foreground object training set including a first image set and position information of foreground objects in each image in the first image set, and the key focus object training set including the first image set, position information of key focus objects in each image in the first image set, and category information of key focus objects in each image in the first image set; a calculation module, used to input the target image into the trained deep neural network, and obtain the position information of the foreground frame, the position information of the key category frame, and the category information corresponding to the key category frame through calculation, wherein the foreground frame is a detection frame that identifies the foreground object, and the key category frame is a detection frame that identifies the key focus object.

[0026] In a fifth aspect, a target tracking device is provided, which includes a processor and a memory, wherein the processor is coupled to the memory, and when the processor executes a computer program or instruction in the memory, the method described in the first aspect and any embodiment thereof is performed.

[0027] In a sixth aspect, a target detection device is provided, wherein the target tracking device comprises a processor and a memory, wherein the processor is coupled to the memory, and when the processor executes a computer program or instruction in the memory, the method described in the second aspect and any embodiment thereof is performed.

[0028] In the seventh aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in a memory from the memory, and executing the method as described in the first aspect and any embodiment thereof, or the method as described in the second aspect and any embodiment thereof.

[0029] In an eighth aspect, a computer-readable storage medium is provided, in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes the method as described in the first aspect and any embodiment thereof, or the method as described in the second aspect and any embodiment thereof.

[0030] In the ninth aspect, a computer program product comprising instructions is provided, which, when executed on a computer or a processor, causes the computer or the processor to execute the method as described in the first aspect and any embodiment thereof, or the method as described in the second aspect and any embodiment thereof.

[0031] The technical effects of the third to ninth aspects can refer to the contents of various possible implementation methods of the first to second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the architecture of a target tracking system provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a target detection method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a detection frame and a key category frame provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of text segmentation and celebrity detection provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a target tracking method provided in an embodiment of the present application Figure 1 ;

[0038] Figure 7 A schematic diagram of a target tracking method provided in an embodiment of the present application Figure 2 ;

[0039] Figure 8 A schematic diagram of a target tracking method provided in an embodiment of the present application Figure 3 ;

[0040] Fig. 9 A schematic diagram of the center distance between an IOU and a frame provided in an embodiment of the present application;

[0041] Fig.10 A schematic diagram of the structure of a target tracking device provided in an embodiment of the present application;

[0042] Fig.11 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media with various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal with one or more data packets (for example, data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0044] like Figure 1 As shown, a target tracking system provided by an embodiment of the present application may include a camera 11, a server 12, and a terminal device (e.g., a virtual reality device 13, a mobile phone 14, a vehicle-mounted device, a drone, etc.) 10. The target tracking method provided by an embodiment of the present application may be run on the cloud side (e.g., the server 12) or on the terminal device side. The target tracking device provided by an embodiment of the present application may be a server 12, a virtual reality device 13, a mobile phone 14, etc.

[0045] Specifically, the camera 11 can transmit the collected video image to the server 12, and the server 12 executes the target tracking method provided in the embodiment of the present application. Alternatively, the terminal device can obtain the video image (for example, the mobile phone 14 can collect the video image through its own camera, and the virtual reality device 13 can obtain the video image from the network), and execute the target tracking method provided in the embodiment of the present application.

[0046] like Figure 2 As shown, taking the terminal device 10 as a mobile phone 14 as an example, the structure of the terminal device 10 is explained.

[0047] The terminal device 10 may include: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a wireless fidelity (Wi-Fi) module 170, a processor 180, a Bluetooth module 181, and a power supply 190 and other components.

[0048] The RF circuit 110 can be used to receive and send signals during the process of sending and receiving information or making calls. It can receive downlink data from the base station and hand it over to the processor 180 for processing; it can send uplink data to the base station. Generally, the RF circuit includes but is not limited to antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer and other devices.

[0049] The memory 120 can be used to store software programs and data. The processor 180 executes various functions and data processing of the terminal device 10 by running the software programs or data stored in the memory 120. The memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 120 stores an operating system that enables the terminal device 10 to run, such as the operating system developed by Apple Inc. Operating system, developed by Google Open source operating system developed by Microsoft Operating system, etc. In the present application, the memory 120 can store an operating system and various application programs, and can also store codes for executing the method of the embodiment of the present application.

[0050] The input unit 130 (e.g., a touch screen) may be used to receive input digital or character information and generate signal inputs related to user settings and function control of the terminal device 10. Specifically, the input unit 130 may include a touch screen 131 disposed on the front of the terminal device 10, which may collect user touch operations on or near it.

[0051] The display unit 140 (i.e., display screen) can be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 10. The display unit 140 may include a display screen 141 disposed on the front of the terminal device 10. The display screen 141 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 140 may be used to display various graphical user interfaces described in this application. The touch screen 131 may be covered on the display screen 141, or the touch screen 131 may be integrated with the display screen 141 to realize the input and output functions of the terminal device 10, and the integrated display screen may be referred to as a touch display screen.

[0052] The terminal device 10 may also include at least one sensor 150, such as a light sensor, a motion sensor, an image sensor (for example, a complementary metal oxide semiconductor (CMOS) sensor). The terminal device 10 may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc. The terminal device 10 can obtain video images through an image sensor.

[0053] The audio circuit 160, the speaker 161, and the microphone 162 may provide an audio interface between the user and the terminal device 10. The audio circuit 160 may transmit the electrical signal converted from the received audio data to the speaker 161, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the RF circuit 110 to be sent to, for example, another terminal, or the audio data is output to the memory 120 for further processing.

[0054] Wi-Fi is a short-range wireless transmission technology. The terminal device 10 can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 170, which provides users with wireless broadband Internet access.

[0055] The Bluetooth module 181 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device 10 can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 181 to exchange data.

[0056] The terminal device 10 also includes a power supply 190 (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0057] The processor 180 is the control center of the terminal device 10. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device 10 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In this application, the processor 180 may refer to one or more processors, and the processor 180 may include one or more processing units; the processor 180 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, the user interface and the application program, and the baseband processor mainly processes wireless communication. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 180. In this application, the processor 180 can run the operating system, the application program, the user interface display and the touch response, as well as the target detection method and the target tracking method involved in the embodiment of the present application. Specifically, the processor 180 can obtain the video image according to the image sensor to track the target, and display the tracking result on the display unit 140.

[0058] As mentioned above, the existing multi-target tracking technology has unstable problems such as jitter and occasional absence when displaying the tracking frame. For end-side application scenarios such as mobile phones and virtual reality devices, this instability problem is unacceptable. The embodiment of the present application provides a target tracking method, which updates the tracking frame by obtaining a detection frame with medium and high stability through target detection, so that the display of the tracking frame is more stable, which can solve the problem of unstable display of the tracking frame in multi-target tracking.

[0059] like Figure 3 As shown, the embodiment of the present application provides a target detection method, including:

[0060] S301. Train a deep neural network according to a foreground object training set and a focused object training set to obtain a trained deep neural network.

[0061] The foreground is relative to the background. The background refers to objects that are not of concern, generally refers to stationary objects, such as the sky, roads, trees, buildings, etc. The foreground object refers to the object of ordinary concern, generally refers to objects that move relative to the background, such as vehicles, pedestrians, etc. The focus category refers to the object of focus.

[0062] The key category box is the detection box that identifies the key focus object.

[0063] The foreground object training set includes a first image set and position information of foreground objects in each image in the first image set, wherein the foreground object in the first image can be identified by a detection frame that identifies the foreground object, namely, a foreground frame, and the position information of the foreground object in the first image is the position information of the foreground frame.

[0064] The focus object training set includes a first image set, position information of the focus object in each image in the first image set, and category information of the focus object in each image in the first image set, wherein the focus object in the first image can be identified by a detection frame identifying the focus object, that is, a focus category frame, for example, a detection frame identifying a face. The position information of the focus object in the first image is the position information of the focus category frame.

[0065] For example, Figure 4 As shown, the detection frame is 41 and the key category frame is 42.

[0066] S302: Input the target image into the trained deep neural network, and obtain the position information of the foreground frame, the position information of the key category frame, and the category information corresponding to the key category frame through calculation.

[0067] That is to say, three types of information can be obtained by inputting a frame of target image into a deep neural network.

[0068] The object detection method provided in the embodiment of the present application combines the category information corresponding to the key category box, decouples classification and recognition, can support more classifications, and its classification accuracy is relatively higher. Therefore, it can also be applied to many scenarios such as image search, celebrity search, friend search, text segmentation, car model recognition, banknote recognition, etc.

[0069] In the prior art, target detection mainly performs foreground detection, that is, only the position information of the foreground object is output, but not the category information of the foreground object, and the edge of the object is required to appear in the picture. This limits the application scenarios of target detection to a certain extent.

[0070] For example, Figure 5 As shown in A, for documents of similar formats, the existing target detection technology cannot detect and identify the target because the target image does not include the edge of the paper, and thus cannot segment the text. The target detection method described in the above application can limit the target category to text, so that the text can be marked by the key category box during target detection, and further segment the text.

[0071] For example, Figure 5 As shown in B, when performing celebrity search or face recognition, the face needs to be identified by a detection frame. The existing target detection technology will output a frame of the entire human body, which affects the accuracy of celebrity search or face recognition. The target detection method described in the above application can further improve the accuracy of celebrity search or face recognition by limiting the target category to a face, so that the face can be identified by a key category frame during target detection.

[0072] like Figure 6As shown, the target tracking method provided in the embodiment of the present application includes S601-S603:

[0073] S601: Perform target detection on a frame of target image to obtain at least one detection frame, and perform tracking detection on the target image to obtain at least one tracking frame.

[0074] The target image may be a frame image in a video. For example, firstly, a video is deframed and framed (e.g., A frame framed at equal intervals) (deframed and framed may be performed while shooting the video) to obtain one of the target images. For other frame images other than the target image, each frame may use an existing target tracking algorithm to obtain a target tracking frame. For the target image, the target tracking method provided in the embodiment of the present application may be used to obtain a target tracking frame.

[0075] It should be noted that the acceleration information of the terminal device where the image sensor is located can be detected, and the frequency of target detection and tracking detection can be adjusted according to the acceleration information. For example, the greater the acceleration of the image sensor, the faster the terminal device moves or the more severe the shaking. The frequency of target detection and tracking detection can be increased to improve the detection accuracy. Otherwise, it means that the terminal device moves slowly or the shaking is not severe. The frequency of target detection and tracking detection can be reduced to reduce power consumption.

[0076] This application does not limit the specific forms of target detection and tracking detection. For example, target detection can adopt existing target detection methods, or adopt Figure 3 The target detection method shown in FIG. 1 can use a lightweight tracking algorithm, such as kernelized correlation filters (KCF), etc. The target detection and tracking detection can be run in different threads to maintain their independence.

[0077] like Figure 7 As shown, performing target detection on a frame of target image to obtain at least one detection frame may include S6011-S6013:

[0078] S6011. Obtain a foreground frame, a focus category frame, and category information corresponding to the focus category frame according to a frame of target image.

[0079] Specifically, the target image can be input into Figure 3 The pre-trained deep neural network can obtain the position information of the foreground box, the position information of the key category box and the category information corresponding to the key category box.

[0080] S6012: Determine whether the key category box corresponding to the target category belongs to the at least one detection box.

[0081] That is, if the detection frame belongs to a key category frame and its corresponding category information is a target category (eg, face), then the key category frame is retained regardless of whether the edge is close to the target image edge.

[0082] S6013: Determine that a foreground frame whose distance from an edge of the target image is greater than a first threshold belongs to the at least one detection frame.

[0083] That is, if the detection frame does not belong to the key category frame and its edge is very close to the edge of the target image (for example, the first threshold may be 5%), the detection frame will not be retained.

[0084] S602: Determine a stability level of the at least one detection frame according to the historical detection frame and the historical tracking frame.

[0085] The historical detection frame is a detection frame obtained by performing target detection on one or more other frames, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames. For example, a system (such as a processing system on the terminal device side or a processing system on the server side) can save the detection frame obtained by the most recent M (such as 3) target detections as a historical detection frame, and save the tracking frame obtained by the most recent N (such as 5) tracking detections as a historical tracking frame.

[0086] Specifically, Figure 8 As shown, step S602 includes:

[0087] S6021. Remove the detection frame that meets the first preset condition from the at least one detection frame to obtain the remaining detection frames.

[0088] The detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a second threshold (for example, 0.2). When performing target detection as described above to output foreground frames and key category frames, the corresponding detection confidence can also be output. The higher the detection confidence, the more accurate the detection frame; a detection frame whose area ratio relative to the target image is less than a third threshold (for example, 1%); repeated detection frames for the same target, and methods for removing repeated detection frames for the same target include but are not limited to non-maximum suppression (NMS), soft non-maximum suppression (Soft-NMS), and other methods.

[0089] The remaining detection frames obtained in the above manner remove detection frames with lower values, thereby reducing the workload of the subsequent matching process with the at least one tracking frame.

[0090] S6022: Match the remaining detection frames with the historical detection frames to obtain the stability level of the remaining detection frames.

[0091] In the initial state, it is assumed that the stability level of all detection boxes is low stability.

[0092] The historical detection frames are matched one by one with the remaining detection frames. The matching method in the embodiment of the present application includes but is not limited to the Hungarian algorithm, etc., which is not limited here.

[0093] If the obtained first matching score (including but not limited to intersection over union (IOU), center distance of the frame, etc.) is greater than a fourth threshold (eg, 0.5), it is determined that the two are matched successfully at one time; otherwise, it is determined that the two are matched unsuccessfully at one time.

[0094] Among them, Fig. 9 As shown in A, IOU can represent the ratio of the intersection and union between two detection boxes. Fig. 9 As shown in B, the center distance of the boxes may represent the distance d between the center points of the two boxes.

[0095] When a remaining detection frame successfully matches a historical detection frame once, if the detection confidence of the remaining detection frame and the detection confidence of the matched historical detection frame are both greater than the fifth threshold, the stability level of the remaining detection frame is determined to be high stability; otherwise, the stability level of the remaining detection frame is determined to be medium stability.

[0096] When a remaining detection frame fails to be successfully matched with any historical detection frame once, the stability level of the remaining detection frame is maintained as low stability.

[0097] S6023: Match the second detection frame with the historical tracking frame to update the stability level of the second detection frame.

[0098] Further, for the remaining detection frames that still maintain low stability after the matching of the above step S6022 (which can be called the second detection frame in the remaining detection frame, the second detection frame is the remaining detection frame determined to be a detection frame with low stability after the matching of step S6022, and the number of the second detection frames can be one or more), the historical detection frame and the second detection frame with low stability are matched one by one. If the obtained second matching score (including but not limited to IOU, center distance of the frame, etc.) is greater than the sixth threshold (for example, 0.3), it is determined that the secondary matching of the two is successful, otherwise it is determined that the secondary matching of the two fails. Among them, the sixth threshold is less than the fourth threshold, that is, the conditions for the secondary matching are looser than the conditions for the primary matching.

[0099] When a second detection frame with low stability successfully matches a historical detection frame for the second time, the stability level of the detection frame with low stability is updated to medium stability. When a second detection frame with low stability fails to successfully match any historical detection frame for the second time, the stability level of the second detection frame is maintained at low stability.

[0100] The purpose of further matching the second detection frame with low stability is to prevent the detection frame from being mistakenly deleted due to misjudgment.

[0101] It should be noted that the order of step S6022 and step S6023 can be swapped, that is, the second detection frame and the historical tracking frame can be matched one by one to obtain the stability level of the second detection frame, and then the second detection frame with low stability can be matched with the historical detection frame to update the stability level of the second detection frame.

[0102] In this way, because the determination of the high stability of the detection frame depends on the confidence of the historical detection frame, if the detection frame is matched with the historical tracking frame first, only the detection frame preliminarily determined to be low stability and medium stability can be obtained. For the detection frame preliminarily determined to be low stability and medium stability (that is, all detection frames), it is necessary to match with the historical detection frame again to finally obtain a detection frame with a more reliable stability level, which will bring redundant calculations (for example, the detection frame with medium stability is matched with both the historical tracking frame and the historical detection frame). Therefore, executing step S6022 first and then executing step S6023 can save computing resources.

[0103] If in step S6022, once the stability level of the current detection frame is determined to be high stability, the current round of matching can be stopped, that is, the current detection frame does not need to continue to match with other historical detection frames.

[0104] In addition, step S6023 may not be executed. However, if step S6023 is not executed, some detection frames with medium stability may be missed, resulting in reduced stability of the tracking frame, making it easier to delete some tracking frames by mistake.

[0105] S603: Match the first detection frame with the at least one tracking frame respectively, and update the target tracking frame displayed in the image according to the matching result.

[0106] The first detection frame does not include a detection frame whose stability level is the first level in the at least one detection frame. The first level is low stability. The first detection frame may refer to multiple detection frames. That is, after step S6022, the detection frames with medium and high stability are retained, and the detection frames with low stability are deleted.

[0107] After the first detection frame is matched with the at least one tracking frame respectively, one matching result is that each detection frame in the first detection frame is not matched with a tracking frame, for example, the detection frame and all tracking frames of the at least one tracking frame do not match; another matching result is that the detection frame can be matched, for example, if the first detection frame in the first detection frame and the first tracking frame in the at least one tracking frame are matched successfully, then the detection frame does not need to be matched with other tracking frames in the at least one tracking frame.

[0108] For matching the first detection frame with the at least one tracking frame respectively:

[0109] That is to say, the detection frame with medium and high stability is matched one by one with the at least one tracking frame. If the obtained third matching score (including but not limited to IOU, center distance of the frame, etc.) is greater than or equal to the seventh threshold (for example, 0.75), the matching result between the first detection frame and the tracking frame is determined to be a high matching degree. If the obtained third matching score is less than the seventh threshold but greater than or equal to the eighth threshold (for example, 0.3), the matching result between the first detection frame and the tracking frame is determined to be a medium matching degree. If the obtained third matching score is less than the eighth threshold, the matching result between the first detection frame and the tracking frame is determined to be a low matching degree. If the first detection frame does not match any tracking frame, the matching result between the detection frame and the at least one tracking frame is determined to be unmatched.

[0110] For updating the target tracking frame displayed in the target image according to the matching result: that is, according to different matching results and the stability level of the first detection frame, different adjustments are made to the tracking frame in the target image.

[0111] Optionally, a new tracking frame may be created in the target image according to a third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched. The second level is high stability.

[0112] That is to say, for the third detection frame with high stability: if the matching results of the third detection frame and the at least one tracking frame are both unmatched, and the number of tracked targets has not reached the upper limit, a new tracking frame is initialized in the target image according to the detection frame.

[0113] In addition, if the matching result between the third detection frame and the at least one tracking frame is a medium matching degree, the at least one tracking frame may be updated according to the detection frame, for example, the at least one tracking frame may be initialized with the detection frame. If the matching result between the third detection frame and the at least one tracking frame is a high matching degree, the at least one tracking frame may not be updated.

[0114] For a detection frame with medium stability, if a matching result between the detection frame and the at least one tracking frame is medium matching or high matching, the at least one tracking frame is not updated.

[0115] Optionally, the first tracking frame displayed in the target image may be suppressed or punished, wherein the first tracking frame is a tracking frame whose matching result is unmatched or with a low matching degree among the at least one tracking frame.

[0116] That is, the tracking frames whose matching results with the first detection frame are no match or low match are suppressed or penalized, for example, the number of suppression times can be accumulated.

[0117] The suppression methods may include edge suppression, overlap suppression, independence suppression, etc. And the tracking frame whose suppression times exceed the suppression threshold may be removed to obtain the target tracking frame.

[0118] Edge suppression. If the distance between the updated tracking frame and the edge of the target image is less than the ninth threshold, the tracking frame is suppressed, for example, the number of suppressions is accumulated. It should be noted that if the distance between the tracking frame and the edge of the target image in two directions (for example, the upper left corner, the upper right corner, the lower left corner, and the lower right corner) is less than the ninth threshold, the tracking frame can be suppressed twice.

[0119] Overlap suppression: If the IOU of two updated tracking boxes is greater than the tenth threshold (e.g., 0.6), one of the tracking boxes is suppressed according to the suppression strategy (e.g., suppressing the smaller tracking box, suppressing the tracking box that appears later, etc.), for example, accumulating the number of suppression times.

[0120] Independence suppression. If the IOU of the overlapping area of ​​two updated tracking boxes exceeds the eleventh threshold (e.g., 0.8) with one of them, one of the tracking boxes is suppressed according to the suppression strategy (e.g., suppressing the smaller tracking box, suppressing the tracking box that appears later, etc.), for example, accumulating the number of suppressions.

[0121] For tracking frames whose suppression times are greater than or equal to the twelfth threshold, they are deleted.

[0122] In addition, after obtaining the target tracking frame, the target tracking smoothing can be performed, that is, the target tracking frame and the historical tracking frame are smoothed and filtered (such as Kalman filtering) to suppress the noise in the tracking frame and make the output smoother and more stable.

[0123] The target tracking method provided by the embodiment of the present application performs target detection on a frame of target image to obtain at least one detection frame, and performs tracking detection on the target image to obtain at least one tracking frame. According to the historical detection frame and the historical tracking frame, the stability level of the at least one detection frame is determined, wherein the historical detection frame is a detection frame obtained by performing target detection on other one or more frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on other one or more frames of images. The first detection frame is matched with the at least one tracking frame respectively, and the target tracking frame displayed in the image is updated according to the matching result, wherein the first detection frame does not include the detection frame with a stability level of the first level in the at least one detection frame. By updating the tracking frame without including the detection frame with high stability of the first level, the display of the tracking frame is made more stable, which can solve the problem of unstable display of the tracking frame in multi-target tracking.

[0124] It can be understood that in each of the above embodiments, the methods and / or steps implemented by the terminal device can also be implemented by components (such as chips or circuits) that can be used in the terminal device.

[0125] Accordingly, the embodiment of the present application also provides a target tracking device and a target detection device, the target tracking device is used to implement the above-mentioned target tracking method, and the target detection device is used to implement the above-mentioned target detection method. The target tracking device or the target detection device can be the terminal device in the above-mentioned method embodiment, or a device including the above-mentioned terminal device, or a chip or functional module in the terminal device. It is understandable that in order to realize the above-mentioned functions, the target tracking device or the target detection device includes a hardware structure and / or software module corresponding to the execution of each function.

[0126] Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0127] The embodiment of the present application can divide the functional modules of the target tracking device or the target detection device according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0128] For example, take the target tracking device as the terminal device in the above method embodiment. Fig.10 FIG. 1 is a schematic diagram showing the structure of a target tracking device 100. The target tracking device 100 includes an acquisition module 1001, a determination module 1002, an update module 1003, and optionally, an adjustment module 1004. The target tracking device 100 can execute Figure 6 , Figure 7 , Figure 8 The target tracking method in the above modules can also be called processing units. The acquisition module 1001 can execute, for example Figure 6 Step S601 in Figure 7 Step S6011, Figure 8 The determination module 1002 may perform, for example, Figure 6 Step S602 in Figure 7 Steps S6012-S6013 in Figure 8 The update module 1003 may perform steps S6021-S6023 in the embodiment of the present invention. Figure 6 Step S603 in Figure 8 Step S603 in .

[0129] Exemplarily, the acquisition module 1001 is used to perform target detection on a frame of target image to obtain at least one detection frame, and perform tracking detection on the target image to obtain at least one tracking frame.

[0130] The determination module 1002 is used to determine the stability level of the at least one detection frame based on the historical detection frame and the historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on one or more other frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames of images.

[0131] The updating module 1003 is used to match the first detection frame with the at least one tracking frame respectively, and update the target tracking frame displayed in the target image according to the matching result, wherein the first detection frame does not include the detection frame with a stability level of the first level in the at least one detection frame.

[0132] In a possible implementation, the first level is low stability.

[0133] In a possible implementation, the determination module 1002 is specifically used to: remove the detection frame that meets the first preset condition from the at least one detection frame to obtain the remaining detection frame; match the remaining detection frame with the historical detection frame to obtain the stability level of the remaining detection frame, the remaining detection frame includes a second detection frame with low stability; match the second detection frame with the historical tracking frame to update the stability level of the second detection frame.

[0134] In a possible implementation, the detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a first threshold; a detection frame whose area ratio relative to the target image is smaller than a second threshold; and a repeated detection frame for the same target.

[0135] In a possible implementation, the updating module 1003 is specifically configured to: create a new tracking frame in the target image according to a third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched.

[0136] In a possible implementation manner, the second level is high stability.

[0137] In a possible implementation, the updating module 1003 is specifically configured to suppress or punish a first tracking frame displayed in the target image, wherein the first tracking frame is a tracking frame whose matching result is unmatched or low matching degree among the at least one tracking frame.

[0138] In a possible implementation, the adjustment module 1004 is configured to adjust the frequency of target detection according to acceleration information.

[0139] In this embodiment, the target tracking device 100 is presented in the form of dividing various functional modules in an integrated manner. The "module" here may refer to a specific ASIC, a circuit, a processor and a memory that executes one or more software or firmware programs, an integrated logic circuit, and / or other devices that can provide the above functions.

[0140] In a simple embodiment, those skilled in the art can imagine that the target tracking device 100 can be used Figure 2 The form of the terminal device 10 is shown.

[0141] for example, Figure 2 The processor 180 in the terminal device 10 shown can call the computer execution instructions stored in the memory 120 to enable the terminal device 10 to execute the target tracking method in the above method embodiment.

[0142] Specifically, Fig.10 The functions / implementation processes of each module in can be Figure 2 The processor 180 in the terminal device 10 shown calls the computer execution instructions stored in the memory 120 to implement.

[0143] Since the target tracking device 100 provided in this embodiment can execute the above-mentioned target tracking method, the technical effects that can be obtained can refer to the above-mentioned method embodiments, which will not be repeated here.

[0144] For example, take the target detection device as the terminal device in the above method embodiment as an example. Fig.11 FIG. 1 is a schematic diagram showing the structure of a target detection device 110. The target detection device 110 includes a training module 1101 and a calculation module 1102. The target detection device 110 can execute Figure 3 The above modules can also be referred to as processing units. The training module 1101 can perform, for example, Figure 3 The calculation module 1102 may perform, for example, Figure 3 Step S302 in .

[0145] Exemplarily, the training module 1101 is used to train the deep neural network according to the foreground object training set and the key focus object training set to obtain a trained deep neural network, wherein the foreground object training set includes the first image set and the position information of the foreground objects in each image in the first image set, and the key focus object training set includes the first image set, the position information of the key focus objects in each image in the first image set, and the category information of the key focus objects in each image in the first image set.

[0146] The calculation module 1102 is used to input the target image into the trained deep neural network, and obtain the position information of the foreground frame, the position information of the key category frame and the category information corresponding to the key category frame through calculation, wherein the foreground frame is a detection frame that identifies the foreground object, and the key category frame is a detection frame that identifies the key focus object.

[0147] In this embodiment, the target detection device 110 is presented in the form of dividing various functional modules in an integrated manner. The "module" here may refer to a specific ASIC, a circuit, a processor and a memory that executes one or more software or firmware programs, an integrated logic circuit, and / or other devices that can provide the above functions.

[0148] In a simple embodiment, those skilled in the art can imagine that the target detection device 110 can adopt Figure 2 The form of the terminal device 10 is shown.

[0149] for example, Figure 2 The processor 180 in the terminal device 10 shown can call the computer execution instructions stored in the memory 120 to enable the terminal device 10 to execute the target detection method in the above method embodiment.

[0150] Specifically, Fig.11 The functions / implementation processes of each module in can be Figure 2 The processor 180 in the terminal device 10 shown calls the computer execution instructions stored in the memory 120 to implement.

[0151] Since the target detection device 110 provided in this embodiment can execute the above target detection method, the technical effects that can be obtained can refer to the above method embodiments and will not be repeated here.

[0152] The embodiment of the present application also provides a target tracking device, which includes a processor and a memory, wherein the processor is coupled to the memory, and when the processor executes a computer program or instruction in the memory, the execution Figure 6 , Figure 7 , Figure 8 target tracking method.

[0153] The present application also provides a target detection device, which includes a processor and a memory. The processor is coupled to the memory. When the processor executes a computer program or instruction in the memory, the processor executes Figure 3 target detection method.

[0154] The present application also provides a chip, including: a processor and an interface, for calling and running a computer program stored in the memory from the memory, and executing Figure 6 , Figure 7 , Figure 8 target tracking method.

[0155] The present application also provides a chip, including: a processor and an interface, for calling and running a computer program stored in the memory from the memory, and executing Figure 3 target detection method.

[0156] The present application also provides a computer-readable storage medium in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes Figure 6 , Figure 7 , Figure 8 target tracking method.

[0157] The present application also provides a computer-readable storage medium in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes Figure 3 target detection method.

[0158] The present application also provides a computer program product including instructions, which, when executed on a computer or processor, causes the computer or processor to execute Figure 6 , Figure 7 , Figure 8 target tracking method.

[0159] The present application also provides a computer program product including instructions, which, when executed on a computer or processor, causes the computer or processor to execute Figure 3 target detection method.

[0160] The present application embodiment provides a chip system, which includes a processor for executing a target tracking device. Figure 6 , Figure 7 , Figure 8 target tracking method.

[0161] The present application provides a chip system, which includes a processor for executing a target detection device. Figure 3 target detection method.

[0162] It should be noted that the chip system also includes a memory, which is used to store necessary program instructions and data for the terminal device. The chip system may include a chip, an integrated circuit, or a chip and other discrete devices, which is not specifically limited in the embodiments of the present application.

[0163] Among them, the target tracking device, target detection device, chip, computer storage medium, computer program product or chip system provided in the present application are all used to execute the method described above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the implementation methods provided above and will not be repeated here.

[0164] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0165] The memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0166] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0167] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target tracking method, characterized in that: include: Performing target detection on a frame of target image to obtain at least one detection frame, and performing tracking detection on the target image to obtain at least one tracking frame; Determining a stability level of the at least one detection frame according to a historical detection frame and a historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on one or more other frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames of images; the stability refers to whether the display of the tracking frame updated by the detection frame is stable, and the stability level includes low stability, medium stability and high stability; The first detection frame is matched with the at least one tracking frame respectively, and the target tracking frame displayed in the target image is updated according to the matching result, wherein the first detection frame does not include a detection frame whose stability level is the first level among the at least one detection frame.

2. The method according to claim 1, characterized in that The first level is low stability.

3. The method according to claim 1 or 2, characterized in that: The determining, according to the historical detection frames and the historical tracking frames, the stability level of the at least one detection frame comprises: Removing the detection frame that satisfies the first preset condition from the at least one detection frame to obtain the remaining detection frame; Matching the remaining detection frames with the historical detection frames to obtain stability levels of the remaining detection frames, wherein the remaining detection frames include a second detection frame with low stability; The second detection frame is matched with the historical tracking frame to update the stability level of the second detection frame.

4. The method according to claim 3, characterized in that The detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a first threshold; a detection frame whose area ratio relative to the target image is smaller than a second threshold; and a repeated detection frame for the same target.

5. The method according to claim 1 or 2, characterized in that: The updating of the tracking frame displayed in the target image according to the matching result includes: A new tracking frame is created in the target image according to a third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched.

6. The method according to claim 5, characterized in that The second level is high stability.

7. The method according to claim 1 or 2, characterized in that: The updating of the tracking frame displayed in the target image according to the matching result includes: A first tracking frame displayed in the target image is suppressed or penalized, wherein the first tracking frame is a tracking frame whose matching result is unmatched or low matching degree among the at least one tracking frame.

8. The method according to claim 1 or 2, characterized in that: The method further includes: adjusting the frequency of the target detection according to acceleration information.

9. A target tracking device, characterized in that: include: An acquisition module, configured to perform target detection on a frame of target image to obtain at least one detection frame, and perform tracking detection on the target image to obtain at least one tracking frame; a determination module, configured to determine a stability level of the at least one detection frame according to a historical detection frame and a historical tracking frame, wherein the historical detection frame is a detection frame obtained by performing target detection on one or more other frames of images, and the historical tracking frame is a tracking frame obtained by performing tracking detection on one or more other frames of images; the stability refers to whether the display of the tracking frame updated by the detection frame is stable, and the stability levels include low stability, medium stability and high stability; An updating module is used to match the first detection frame with the at least one tracking frame respectively, and update the target tracking frame displayed in the target image according to the matching result, wherein the first detection frame does not include a detection frame with a stability level of the first level in the at least one detection frame.

10. The device according to claim 9, characterized in that The first level is low stability.

11. The device according to claim 9 or 10, characterized in that The determining module is specifically used for: Removing the detection frame that satisfies the first preset condition from the at least one detection frame to obtain the remaining detection frame; Matching the remaining detection frames with the historical detection frames to obtain stability levels of the remaining detection frames, wherein the remaining detection frames include a second detection frame with low stability; The second detection frame is matched with the historical tracking frame to update the stability level of the second detection frame.

12. The device according to claim 11, characterized in that The detection frame that meets the first preset condition includes at least one of the following: a detection frame whose detection confidence is lower than a first threshold; a detection frame whose area ratio relative to the target image is smaller than a second threshold; and a repeated detection frame for the same target.

13. The device according to claim 9 or 10, characterized in that The update module is specifically used for: A new tracking frame is created in the target image according to a third detection frame, wherein the third detection frame is a detection frame in the first detection frame whose stability level is the second level and whose matching result is unmatched.

14. The device according to claim 13, characterized in that The second level is high stability.

15. The device according to claim 9 or 10, characterized in that The update module is specifically used for: A first tracking frame displayed in the target image is suppressed or penalized, wherein the first tracking frame is a tracking frame whose matching result is unmatched or low matching degree among the at least one tracking frame.

16. The device according to claim 9 or 10, characterized in that Also includes: The adjustment module is used to adjust the frequency of the target detection according to the acceleration information.

Images