Embedded multi-target detection and single-target tracking method and system, storage medium and electronic equipment

Abstract

The invention discloses an embedded multi-target detection and single-target tracking method and system, a storage medium and electronic equipment, and the method comprises the steps: carrying out the detection of all targets in an obtained image through a target detector, and obtaining a detection frame; numbering and continuously tracking the targets in the detection frame through a tracker; the detected detection frame and the corresponding frame picture are transmitted to a target selection module for shunting and screening processing; based on a target selection module, the selected tracking result is input into a tracking feedback module, and the tracking feedback module judges whether the tracking result of the current frame is matched with the tracking result of the previous frame in real time, adjusts the tracking state according to the matching result and feeds back the tracking state to the target selection module; after model conversion and compilation, the algorithm is deployed to an embedded platform to realize automatic detection and tracking of a target. According to the method, all targets can be detected and prompted in real time in the video, and in the aspect of calculation efficiency, a selective tracking mechanism is adopted, so that the real-time response capability of the system is improved.

Description

Technical Field

[0001] This invention relates to an embedded multi-target detection and single-target tracking method, system, storage medium, and electronic device. Specifically, it relates to an automatic multi-target and single-target detection and selective tracking method, system, storage medium, and electronic device based on an embedded platform, belonging to the field of target detection and tracking technology. Background Technology

[0002] Generally speaking, in the field of computer vision, target recognition and tracking technology is one of the core technologies in applications such as video analysis, autonomous driving, and intelligent surveillance. Current target recognition and tracking technologies are mainly divided into two categories: single-target recognition and tracking (SOT) and multi-target recognition and tracking (MOT), each with its own advantages and limitations.

[0003] Single-target recognition and tracking techniques (such as KCF and CSRT) require manually initializing the bounding box to select the target of interest and then continuously tracking that target in subsequent frames. Typical applications of this type of method include long-term monitoring of specific targets and motion analysis. While performing well in scenarios involving continuous tracking of specific targets,

[0004] However, it has three major drawbacks: First, the initial position of the target needs to be manually given; without a host computer or other input device, automatic tracking is impossible. Second, when the target is occluded or temporarily disappears, the tracker is prone to losing sight of the target, leading to tracking failure. Finally, this method cannot simultaneously monitor or provide prompts for other potentially interesting targets in the video, limiting its application scope. Therefore, single-target tracking technology often struggles to handle multi-target interactions and dynamic changes in complex scenarios in practical applications.

[0005] Multi-target recognition and tracking technologies typically consist of a cascaded array of target detectors and trackers to achieve full target tracking. The target detector (such as YOLO, SSD, etc.) is responsible for detecting all possible targets in each frame, while the tracker (such as DeepSORT, SORT, etc.) is responsible for continuously tracking these targets. This type of method can identify and track all targets in a video and is suitable for scenarios with frequent multi-target interactions, such as traffic monitoring and crowd analysis.

[0006] However, multi-target tracking technology also has its problems: First, it suffers from computational redundancy. Because it requires detecting and tracking all targets, the computational complexity is high, especially on edge computing devices with limited computing power, making it difficult to guarantee real-time performance. Second, it lacks flexibility. In practical applications, it is usually not necessary to continuously track all targets; instead, it is sufficient to track specific targets while simultaneously detecting and alerting others. Therefore, multi-target tracking appears redundant in scenarios with limited computing resources or high real-time tracking requirements.

[0007] Existing patented technologies still have several key shortcomings. Analysis of current mainstream patents reveals that while traditional YOLO+DeepSORT combination schemes (such as CN114723786B) improve feature fusion, they still require feature extraction and matching for all detected targets, resulting in enormous computational demands and making it difficult to meet real-time requirements on embedded devices. More seriously, such full-scale tracking schemes often lead to resource waste in practical applications, because most scenarios only require focusing on specific targets rather than all targets.

[0008] Existing technologies exhibit significant shortcomings in target tracking robustness. Single-target tracking methods (such as KCF and CSRT) are prone to losing tracking and cannot automatically recover when the target is occluded or briefly disappears. While Transformer-based tracking schemes (such as CN118247314B) optimize feature matching, their ability to recover targets after prolonged occlusion is limited, typically with a recovery window of less than 10 frames. Furthermore, existing patents generally lack specific optimizations for embedded devices. For example, while CN202210939030.3 proposes dehazing to improve detection accuracy, it fails to consider the computational limitations of edge devices, thus restricting practical applications. Summary of the Invention

[0009] To address the numerous shortcomings and deficiencies in the aforementioned background technology, this invention has made improvements and innovations, aiming to provide a safe and reliable embedded multi-target detection and single-target tracking method, system, storage medium, and electronic device. This device can perform real-time detection and prompting of all targets in a video, while continuously tracking a specific target specified by the user. In terms of computational efficiency, this invention innovatively adopts a selective tracking mechanism, which significantly reduces the overall computational overhead of the algorithm and improves the real-time response capability of the system by performing the complete tracking process only on key targets.

[0010] Another objective of this invention is to add a tracking status feedback function, which executes different tracking strategies according to different tracking states. This not only improves the robustness and practicality of the tracking algorithm, but also enhances its adaptability and improves the overall practicality of the system.

[0011] Another objective of this invention is to optimize steps such as distance calculation, further reduce the amount of computation, and thereby improve the overall performance and efficiency of the system.

[0012] To solve the above problems and achieve the above-mentioned objectives, the present invention provides an embedded multi-target detection and single-target tracking method, system, storage medium, and electronic device by adopting the following design structure and the following technical solution:

[0013] An embedded multi-target detection and single-target tracking method includes:

[0014] S1, the target detector detects all targets in the acquired image to obtain the detection box;

[0015] S2, uses a tracker to number and continuously track targets within the detection frame;

[0016] S3, the detected detection box and corresponding frame image are sent to the target selection module for splitting and filtering;

[0017] S4, based on the target selection module, the selected tracking result is input into the tracking feedback module. This module determines in real time whether the tracking result of the current frame matches the tracking result of the previous frame, and adjusts the tracking state according to the matching result, and feeds it back to the target selection module.

[0018] After model conversion and compilation, the S5 algorithm is deployed to an embedded platform to achieve automatic target detection and tracking.

[0019] Preferably, a multi-target detection and tracking system for implementing the multi-target detection and single-target tracking method includes:

[0020] An object detector is used to detect multiple objects in an image and outputs the location and category information of the detection boxes;

[0021] A tracker is used to number the input detection boxes and associate the same target in different frames using these numbers;

[0022] The target selection module is used to manually select or automatically select specific detection boxes to input into the tracker based on different states;

[0023] The tracking feedback module is used to compare and analyze the current tracking results, tracking status, and historical tracking information to determine and transfer the tracking status. If the system is in a target lock state, this module is also responsible for updating the location information of the locked target and feeding it back to the target selection module to provide a reference for target selection in the next frame.

[0024] Preferably, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the multi-target detection and single-target tracking method according to any one of claims 1 to 7.

[0025] Preferably, an electronic device includes:

[0026] processor;

[0027] Memory for storing the executable instructions of the processor;

[0028] The processor is configured to execute the multi-target detection and single-target tracking method by executing the executable instructions.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] 1. This invention has the advantage of real-time multi-target detection and tracking: it can detect and prompt all targets in the video stream in real time, and continuously track specific targets specified by the user. It combines the advantages of multi-target detection and single-target tracking, and adapts to the needs of different application scenarios.

[0031] 2. The present invention has the advantage of selective tracking mechanism to improve computational efficiency: by performing a complete tracking process only on key targets, the overall computational overhead is greatly reduced, the real-time performance and computational efficiency of the system are improved, and it is particularly suitable for resource-constrained embedded platforms;

[0032] 3. This invention has the advantage of enhanced robustness through tracking status feedback: it automatically adjusts the tracking strategy according to the target's tracking status (locked, unlocked, searching), ensuring that the system can quickly resume tracking when the target is lost or occluded, which greatly improves the robustness and applicability of the tracking algorithm;

[0033] 4. This invention has the advantage of being optimized for embedded scenarios: In resource-constrained embedded platforms, single-target tracking in lock mode significantly reduces the computational load, dynamically allocates computing resources, optimizes the balance between accuracy and energy efficiency, and adapts to the computing power limitations of edge devices.

[0034] 5. This invention has the advantages of modular design and flexibility: It adopts a universal modular design, is compatible with a variety of detectors and trackers, and users can easily adapt to different application scenarios by configuring parameters according to actual needs, which significantly improves the flexibility and scalability of the system.

[0035] 6. The present invention has the advantage of a three-state collaborative working mechanism: through the collaborative work of target locking, unlocked and search states, the system can effectively cope with long-term target monitoring (such as security monitoring, wildlife observation) and dynamic target selection scenarios (such as drone tracking, intelligent follow-up shooting), avoiding target loss and resource waste in traditional solutions, realizing intelligent tracking management, the locking mode ensures continuous tracking of core targets, the search mode deals with target loss anomalies, and the unlocked mode provides a flexible initialization interface.

[0036] 7. The present invention has the advantages of core innovation and technological breakthrough: innovative selective tracking paradigm, long-term retrieval mechanism (60-frame search window), embedded optimization technology (innovative memory management and computing methods) and general modular design, which make the present invention significantly superior to the existing technology in terms of computing efficiency, robustness, flexibility and embedded adaptability.

[0037] 8. This invention effectively solves the inherent defects of traditional solutions through multi-mode design, including the inability of single-target trackers to cope with target loss and the waste of resources caused by full multi-target tracking;

[0038] 9. Compared with traditional fixed-mode trackers, this invention has significant advantages: it implements a scene-adaptive tracking strategy that can automatically switch working modes based on target visibility; it provides a personalized interactive design and supports multiple locking methods; it establishes a configurable parameter system, allowing key parameters such as locking duration and target loss search window length to be dynamically adjusted according to actual needs; at the same time, this intelligent multi-mode design not only solves the adaptability problem of traditional solutions in complex scenarios, but also significantly improves the practicality and reliability of the system through fine-grained state management, enabling it to better meet diverse application needs, especially achieving the best balance between performance and energy efficiency on edge devices with limited computing resources. Attached Figure Description

[0039] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0040] Figure 1 This is an overall workflow diagram of the present invention;

[0041] Figure 2 This describes the workflow of the target selection module of the present invention under different tracking modes;

[0042] Figure 3 This describes the workflow of the tracking feedback module of the present invention under different tracking modes;

[0043] Figure 4 This is a schematic diagram illustrating the implementation effect of the present invention. Detailed Implementation

[0044] To make the technical means, inventive features, objectives, and effects of this invention readily understandable, the technical solution of this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other. The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] As per the instruction manual Figures 1 to 4 As shown, a method for multi-target automatic detection and selective tracking system based on an embedded platform includes:

[0047] S1, the target detector detects all targets in the acquired image to obtain the detection box;

[0048] S2, uses a tracker to number and continuously track targets within the detection frame;

[0049] S3, the detected detection box and corresponding frame image are sent to the target selection module for splitting and filtering;

[0050] S4, based on the target selection module, the selected tracking result is input into the tracking feedback module. This module determines in real time whether the tracking result of the current frame matches the tracking result of the previous frame, and adjusts the tracking state according to the matching result, and feeds it back to the target selection module.

[0051] After model conversion and compilation, the S5 algorithm is deployed to an embedded platform to achieve automatic target detection and tracking.

[0052] In this invention, the target detection and tracking process can be fully automated, eliminating the need for manual selection of the target unless there are special requirements.

[0053] Furthermore, in step S3, the target selection module performs a detection box splitting process based on the current tracking state, including:

[0054] The target selection module counts the number of target boxes detected by the detector. If the number of target boxes received by the target selection module is 0, that is, no target is detected in the current frame, the tracking of the current frame will end directly and the detection result of the next frame will be received.

[0055] If the number of target boxes is not 0, the target box selection stage will be entered, and the tracking will be split according to the current tracking status.

[0056] Furthermore, in step S3, the target selection module performs a filtering process for detection boxes based on the current tracking state, including:

[0057] S301, If ​​the current system state is target locked, traverse all detected target boxes in the current frame, using the center pixel position of the tracked target box in the previous frame as the origin, and calculate the distance between the center of the detected target box and the center of the tracked target box. The sorting is performed, and the distance calculation adopts the square form of the L2 norm of the two-dimensional plane to avoid square root operation and improve real-time performance. This process is expressed as equation (1):

[0058]

[0059] in, For the traversal of the current frame, the first... The center coordinates of the detected target bounding box For the first The x-coordinate of the center of the detection target bounding box For the first The ordinate of the center of each detection target bounding box. Use the reference coordinates (the target box or center of the image in the previous frame). This refers to the x-coordinate of the center of the tracking frame in the previous frame or the x-coordinate of the center of the image. The vertical coordinate of the center of the tracking box in the previous frame or the vertical coordinate of the center of the screen is used. Finally, the detection target box that is closest to the tracked object is input into the target tracker.

[0060] S302, If the current tracking status is target not locked, it means that the current tracking object has not been initialized. By default, the detection target box with the shortest Euclidean distance from the center of the screen is selected as the input target tracker. Alternatively, a detection target box can be manually specified as the initial tracking target box and input into the target tracker.

[0061] S303, If the current tracking state is target loss search, then all received target bounding boxes are input into the target tracker to expand the search range to cope with the reappearance of the target after a brief occlusion. Tracking state diversion strategy. Represented as equation (2):

[0062]

[0063] in, This represents the set of all detected targets in the current frame. This describes how to select the appropriate target from among these targets based on the three different tracking states [Locked, Unlocked, Searching] to enter the tracker. Indicates the first in the current frame Detection box The center coordinates, Indicates the target box that was locked in the previous frame. The center coordinates, This indicates the center coordinates of the video frame. These coordinates are constant values ​​and are fixed based on the size of the video frame.

[0064] In this invention, the detection bounding box closest to the tracked object is input into the target tracker. For example, in the target-locked state, the distances between the n target positions detected in frame t+1 and the locked target in frame t are respectively... … Then select distance The smallest target enters the tracker;

[0065] The target selection in step S3 can reduce the feature calculation of the tracker and reduce the overall latency of detection and tracking. For example, in a video frame containing an average of 20 targets, only one target is selected for the tracker in the target locked or unlocked state. Compared with traditional full tracking, the tracking feature calculation is reduced to 1 / 20. On an embedded platform with an NPU computing power of 2 TOPS, the tracking latency is reduced from 100ms to 5ms, which greatly improves the real-time performance of target tracking.

[0066] Furthermore, in step S4, the tracking feedback module performs a discrimination process based on the current tracking status, including:

[0067] S401, When tracking starts, the target is in an unlocked state by default. In this state, the tracking feedback module will determine whether the tracked object has been initialized. If it has not been initialized, the new tracked object will be initialized by the ID and position information of a tracking box output by the tracker. The initialization state is determined by equation (3):

[0068]

[0069] InitStatus represents the initialization status; it is set to True if the target has been initialized, and False if it has not been initialized. This indicates the initialized target information, including the target bounding box location, category, and target ID. This indicates that the value is not initialized.

[0070] S402, If the tracking object has been initialized, determine whether the current tracking frame ID matches the initialized tracking object ID; if the IDs do not match, use the latest tracking frame to reinitialize the tracking object ID and position information. The target update rule in the unlocked state is expressed by equation (4):

[0071]

[0072] in, A unique identifier representing an initialized target. Indicates candidate target information for the current frame. This represents the tracking ID of the candidate target in the current frame; if the ID matches, the initial matching frame counter is incremented by one frame and the position information of the tracked object is updated, as shown in equation (5):

[0073]

[0074] in, This represents the number of consecutive matching frames at frame t. This indicates the number of consecutive matching frames at frame t-1. Once a certain threshold is reached, a state transition will be triggered.

[0075] S403, determine whether the number of matching frames has reached the set standard. If the number of matching frames reaches the standard, enter the target locking state. If it does not reach the standard, end the tracking feedback of the current frame and continue to judge the next frame. This process is represented by equation (6):

[0076]

[0077] in, Indicates the tracking status for the next frame. This represents the minimum number of consecutive matching frames required to enter the locked state.

[0078] Furthermore, in step S4, the tracking feedback module performs a discrimination process based on the current tracking status, which also includes:

[0079] S404, If the system enters the target locking state, it determines whether the latest tracking target ID is consistent with the locked target ID; if they are consistent, it means that the tracking target in the current frame is the same as the tracking target in the previous frame, and the locked target position information can be updated for the target selection module to refer to when selecting the target box in the next frame. Then the tracking result is output. This process is represented by equation (7):

[0080]

[0081] in, This represents the target location information in frame t. This represents the target ID stored in the locked state; if the current target ID is inconsistent, it means that the target is lost, and the system will unlock the target and enter the lost target search state. The overall state transition process is represented by equation (8):

[0082]

[0083] S405, In the search state, the tracking feedback module will traverse all the tracking targets output by the tracker and determine whether the latest target IDs are consistent with the tracking object IDs before the target was lost. The process is represented by equation (9):

[0084]

[0085] in, Indicates whether to retrieve the lost target during the search process. This indicates the total number of tracked targets in the current frame. This represents the ID of the i-th tracked target in frame t; this mechanism enables rapid recovery when the target reappears after occlusion, avoiding tracking drift caused by reinitialization;

[0086] S406, During the search process, if a target with the same ID as the target locked before the target was lost exists, it means that the lost target has been found, the system will re-enter the locking state, the total number of lost frames will be set to zero, the locked target information will be updated, and the target will continue to be tracked; if all targets are not matched after traversal, it means that no tracked target has been found in the current frame, the search state will continue, and the lost frame counter will be incremented by one frame. This process is shown in equation (10):

[0087]

[0088] in, This represents the cumulative number of frames lost by the tracked target at frame t in search mode; This represents the number of frames the tracked target lost in frame t-1;

[0089] S407, In the search state, after each frame search is completed, it is determined whether the total number of lost frames has reached the target lost frame limit. If it has not reached the limit, the current frame tracking feedback ends. If it has reached the limit, the previously tracked object is discarded and the target is re-entered into the target unlocked state. The search state transition process is represented by equation (11):

[0090]

[0091] in, This parameter sets a pre-defined upper limit on the number of frames a target can be lost in search mode. It ensures that the system automatically resets when a target is lost for an extended period, preventing prolonged ineffective searches from consuming computing resources.

[0092] Furthermore, the target detector is any one of YOLO, SSD, or DETR.

[0093] Furthermore, the tracker is either DeepSORT or ByteTrack.

[0094] Specifically, this method is applied on an embedded edge computing platform with an NPU computing power of 2 TOPS, using the YOLO11 model as the target detector and the DeepSORT model as the tracker, with an average processing time of 50 milliseconds per frame.

[0095] In this invention, both the target selection module and the tracking feedback module have three working states: target locked, target unlocked, and target search for lost targets. The target selection module and the tracking feedback module adopt a standardized interface design, which can be integrated with various mainstream target detectors and trackers without modifying the underlying architecture, so as to quickly adapt to different algorithm combinations.

[0096] In this invention, the method is applicable to edge computing platforms and can realize real-time multi-target detection and single-target tracking on devices with limited computing resources.

[0097] Example 2

[0098] The exemplary embodiments disclosed in this invention also provide an embedded multi-target detection and single-target tracking system, the system comprising:

[0099] An object detector is used to detect multiple objects in an image and outputs the location and category information of the detection boxes;

[0100] A tracker is used to number the input detection boxes and associate the same target in different frames using these numbers;

[0101] The target selection module is used to manually select or automatically select specific detection boxes to input into the tracker based on different states;

[0102] The tracking feedback module is used to compare and analyze the current tracking results, tracking status, and historical tracking information to determine and transfer the tracking status. If the system is in a target lock state, this module is also responsible for updating the location information of the locked target and feeding it back to the target selection module to provide a reference for target selection in the next frame.

[0103] Example 3

[0104] The exemplary embodiments disclosed in this invention also provide a computer-readable storage medium that can be implemented as a program product including program code. When the program product is run on an electronic device, the program code is used to cause the electronic device to perform the steps of various exemplary embodiments according to this disclosure described in the "Exemplary Method" section of Embodiment 1 above.

[0105] In one alternative implementation, the program product may be implemented as a portable compact disc read-only memory (CD-ROM) and include program code, and may run on an electronic device, such as a personal computer. However, the program product disclosed herein is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0107] Example 4

[0108] The exemplary embodiments disclosed in this invention also provide an electronic device, which may include a processor and a memory. The memory stores executable instructions of the processor, which may be program code. The processor executes the executable instructions to perform the method in this exemplary embodiment.

[0109] Example 5

[0110] More specifically, an embedded multi-target detection and single-target tracking method includes the following steps:

[0111] Step 1: Construct a basic framework for multi-target detection and tracking. Based on the YOLO target detector, a target detection module is built to process the input video frames in real time, outputting a set of bounding boxes containing target location coordinates and category information. The detection results are input into the DeepSORT multi-target tracker, which assigns a unique identifier to each detected target and establishes a cross-frame target association mechanism to ensure that the same target maintains consistent identification across different video frames. This basic framework provides complete multi-target detection and tracking capabilities for subsequent selective tracking.

[0112] Step 2: Design an intelligent target selection module. Insert the target selection module between the detector and the tracker. This module dynamically adjusts the target selection strategy according to the current tracking status of the system: 1) When the target is locked, select the target box closest to the center point of the tracked target in the previous frame using Euclidean distance as a metric; 2) When the target is unlocked, select the candidate target closest to the center of the image; 3) When the target is lost and searching, send all detection results to the tracker for full matching. This module achieves intelligent selection from multiple targets to specific targets through a state-aware selection mechanism.

[0113] Step 3: Develop a tracking status feedback module and deploy it at the tracker output to monitor the matching of tracking results with historical data in real time. This module is configured with a three-state machine (locked / unlocked / search), and the state transitions are implemented through the following logic: 1) When the system is in the target unlocked state, if the tracked object is not initialized, the number and position information of the first valid tracking frame are used as the initial tracked object; if it has been initialized but the tracking number of the current frame does not match the initialization number, the latest tracking frame is used for re-initialization; if the numbers match, the matching frame counter is incremented, and when the number of consecutive matching frames reaches a certain value, the system enters the locked state; 2) In the locked state, when the current tracked target number matches the locked target number, the locked state is maintained and the target position is updated; when the numbers do not match, the system immediately enters the search state; 3) In the search state, all tracked targets are traversed. If a target with the same number as before it was lost is found, the locked state is restored; if all numbers do not match, the lost frame counter is incremented, and when the number of consecutive lost frames exceeds a certain threshold, the system enters the target unlocked state and resets the target selection module and the tracker.

[0114] The tracking status feedback module and the target selection module form a closed-loop control, which together improves the system robustness.

[0115] Step 4, System Integration and Performance Optimization: Integrate the detector, target selection module, tracker, and tracking feedback module into a unified system, designing a unified interface to achieve compatibility with different detectors and trackers; perform specific optimizations for edge devices, including: 1) using squared distance calculation instead of Euclidean distance to reduce computational load; 2) customizing state transition thresholds, allowing users to freely define the threshold for the number of matching frames required to transition from an unlocked state to a locked state, and the threshold for the number of non-matching frames required to transition from a search state to an unlocked state, reducing wasted computational resources; 3) implementing dynamic management of tracker memory; ultimately forming a complete solution that balances real-time performance and tracking accuracy.

[0116] Example 6

[0117] More specifically, the workflow for single-target tracking in multi-target detection includes:

[0118] Step 1: The video stream is first input to a general object detector, such as YOLO, SSD, or DETR. This invention uses YOLO11 as the basic detector. The detected object bounding box is represented by the pixel coordinates of the upper left and lower right corners of the rectangle and the object category. The detection result of a frame may be no object or one or more objects.

[0119] Step 2: The detected target bounding boxes and corresponding frame images are sent to the target selection module proposed in this invention for sorting and filtering; the workflow of the target selection module is as follows: Figure 2 .

[0120] Further explanation of the target selection module's processing procedure includes:

[0121] Step 21: Count the target boxes detected by the detector. If the number of target boxes received by the target selection module is 0, that is, no target is detected in the current frame, the tracking of the current frame will end directly and the detection result of the next frame will be received. If the number of target boxes is not 0, the target box selection stage will be entered and the tracking will be split according to the different tracking states.

[0122] Step 22: If the current system state is target locked, traverse all detected target boxes in the current frame, using the center pixel position of the tracked target box in the previous frame as the origin, and calculate the distance between the center of the detected target box and the center of the tracked target box. The sorting is performed, and the distance calculation adopts the square form of the L2 norm of the two-dimensional plane to avoid square root operation and improve real-time performance. This process is expressed as equation (1):

[0123]

[0124] in, For the traversal of the current frame, the first... The center coordinates of the detected target bounding box For the first The x-coordinate of the center of the detection target bounding box For the first The ordinate of the center of each detection target bounding box. Use the reference coordinates (the target bounding box or the center of the frame in the previous frame) This refers to the x-coordinate of the center of the tracking frame in the previous frame or the x-coordinate of the center of the image. The coordinates are either the center coordinates of the tracking box in the previous frame or the center coordinates of the image. Finally, the closest detected target box to the tracked object is input into the target tracker. For example, in target-locked mode, the distances between the n target positions detected in frame t+1 and the locked target in frame t are respectively... … Then select distance The smallest target enters the tracker;

[0125] Step 23: If the current tracking status is "target not locked", it means that the current tracking object has not been initialized yet. By default, the detection target box with the shortest Euclidean distance from the center of the screen is selected and input into the target tracker. Alternatively, a detection target box can be manually specified as the initial tracking target box and input into the target tracker.

[0126] Step 24: If the current tracking state is target loss search, input all received target bounding boxes into the target tracker to expand the search range to cope with the reappearance of the target after brief occlusion, and implement a tracking state diversion strategy. Represented as equation (2):

[0127]

[0128] in This represents the set of all detected targets in the current frame. This describes how to select the appropriate target from among these targets based on the three different tracking states [Locked, Unlocked, Searching] to enter the tracker. Indicates the first in the current frame Detection box The center coordinates, Indicates the target box that was locked in the previous frame. The center coordinates, This indicates the center coordinates of the video frame. These coordinates are constant values ​​and are fixed based on the size of the video frame.

[0129] Step 2, target selection, reduces the amount of feature computation required by the tracker and lowers the overall latency of detection and tracking. For example, in a video frame containing an average of 20 targets, only one target is selected for the tracker, whether the target is locked or unlocked. Compared to traditional full tracking, the amount of feature computation is reduced to 1 / 20. On an embedded platform with an NPU computing power of 2 TOPS, the tracking latency is reduced from 100ms to 5ms, significantly improving the real-time performance of target tracking.

[0130] Step 3: Input the target bounding boxes generated in Step 2 into a general target tracker, which is responsible for associating the same target in different frames to achieve the effect of continuous target tracking. For example, DeepSORT, ByteTrack, etc. This invention uses DeepSORT as the basic tracker. The tracker will assign a number to each input target bounding box. If the target numbers in different frames are the same, it means that the target bounding box selects the same target. If a tracked target is occluded or temporarily lost, causing the detector to fail to detect the target, the tracker will retain the target's number and feature information for a certain number of frames. After the target is lost for a certain number of frames, the target information will be permanently deleted.

[0131] Step 4: Input the tracking results output in Step 3 into the tracking feedback module proposed in this invention. This module compares and analyzes the current tracking results, tracking status, and historical tracking information to determine and transfer the tracking status. If the system is in a target-locked state, this module is also responsible for updating the position information of the locked target and feeding it back to the target selection module, providing a reference for target selection in the next frame. Its workflow is as follows: Figure 3 .

[0132] Further explanation of the tracking and feedback module's working process includes:

[0133] Step 41: At the start of tracking, the target is in an unlocked state by default. In this state, the tracking feedback module will determine whether the tracked object has been initialized. If it has not been initialized, the new tracked object will be initialized by the ID and position information of a tracking box output by the tracker. The initialization state is determined by equation (3):

[0134]

[0135] InitStatus represents the initialization status; it is set to True if the target has been initialized, and False if it has not been initialized. This indicates the initialized target information, including the target bounding box location, category, and target ID. This indicates that the value is not initialized.

[0136] Step 42: If the tracking object has been initialized, determine whether the current tracking frame ID matches the initialized tracking object ID; if the IDs do not match, re-initialize the tracking object ID and position information using the latest tracking frame. The target update rule in the unlocked state is expressed by equation (4):

[0137]

[0138] in, A unique identifier representing an initialized target. Indicates candidate target information for the current frame. This represents the tracking ID of the candidate target in the current frame; if the ID matches, the initial matching frame counter is incremented by one frame and the position information of the tracked object is updated, as shown in equation (5):

[0139]

[0140] in, This represents the number of consecutive matching frames at frame t. This indicates the number of consecutive matching frames at frame t-1. Once a certain threshold is reached, a state transition will be triggered.

[0141] Step 43: Determine if the number of matching frames has reached the set standard. If the number of matching frames has reached the standard, enter the target locking state. If it has not reached the standard, end the tracking feedback of the current frame and continue to judge the next frame. This process is represented by equation (6):

[0142]

[0143] in, Indicates the tracking status for the next frame. This represents the minimum number of consecutive matching frames required to enter the locked state.

[0144] Furthermore, in step S4, the tracking feedback module performs a discrimination process based on the current tracking status, which also includes:

[0145] Step 44: If the system enters the target locking state, it determines whether the latest tracking target ID is consistent with the locked target ID; if they are consistent, it means that the tracking target in the current frame is the same target as the tracking target in the previous frame, and the locked target position information can be updated for the target selection module to refer to when selecting the target box in the next frame. Then the tracking result is output. This process is represented by equation (7):

[0146]

[0147] in, This represents the target location information in frame t. This represents the target ID stored in the locked state; if the current target ID is inconsistent, it means that the target is lost, and the system will unlock the target and enter the lost target search state. The overall state transition process is represented by equation (8):

[0148]

[0149] Step 45: In the search state, the tracking feedback module will traverse all the tracked targets output by the tracker and determine whether the latest target IDs are consistent with the tracked object IDs before the target was lost. The process is represented by equation (9):

[0150]

[0151] in, Indicates whether to retrieve the lost target during the search process. This indicates the total number of tracked targets in the current frame. This represents the ID of the i-th tracked target in frame t; this mechanism enables rapid recovery when the target reappears after occlusion, avoiding tracking drift caused by reinitialization;

[0152] Step 46: During the search process, if a target with the same ID as the target locked before the target was lost exists, it means that the lost target has been found. The system will re-enter the locking state, the total number of lost frames will be set to zero, the locked target information will be updated, and the target will continue to be tracked. If all targets are not matched after traversal, it means that no tracked target has been found in the current frame, and the search state will continue. The lost frame counter will be incremented by one frame. This process is shown in Equation (10):

[0153]

[0154] in, This represents the cumulative number of frames lost by the tracked target at frame t in search mode; This represents the number of frames the tracked target lost in frame t-1;

[0155] Step 47: After completing each frame search in the search state, determine whether the total number of lost frames has reached the target lost frame limit. If it has not reached the limit, end the current frame tracking feedback. If it has reached the limit, discard the previously tracked object and re-enter the target unlocked state. The search state transition process is represented by equation (11):

[0156]

[0157] in, This parameter sets a pre-defined upper limit on the number of frames a target can be lost in search mode. It ensures that the system automatically resets when a target is lost for an extended period, preventing prolonged ineffective searches from consuming computing resources.

[0158] For example, the status feedback module of the present invention is set =60, target locked If the image is briefly obscured or goes out of frame T (T<60), the invention will automatically enter the target search state. It provides clear status feedback; if the target reappears within 60 frames, it automatically re-acquires the target and re-enters the target lock state. If the target does not reappear for more than 60 frames, it enters the target unlocked state. It automatically searches for new targets and begins the next round of tracking. This status feedback mechanism improves the stability of target tracking, effectively addressing the temporary or complete disappearance of targets. Furthermore, in practical applications, it provides users with more intuitive and clear feedback, making the tracking status clear and facilitating immediate access to tracking information.

[0159] Previous tracking solutions, whether full-scale multi-target tracking or single-target tracking, did not specifically address the issue of targets temporarily disappearing and then reappearing or completely disappearing. This parameter is intended to handle the temporary disappearance or permanent loss of the target. It also lacks tracking status feedback for the three scenarios that may occur in real-world applications: [Locked], [Unlocked], and [Searching], resulting in an unclear tracking status and making it difficult for users to immediately identify the current tracking situation.

[0160] Specifically, this method is applied on an embedded edge computing platform with an NPU computing power of 2 TOPS, using the YOLO11 model as the target detector and the DeepSORT model as the tracker, with an average processing time of 50 milliseconds per frame.

[0161] This invention relates to an embedded multi-target detection and single-target tracking method, system, storage medium, and electronic device. Referring to the scheme of directly cascading a multi-target detector with a multi-target tracker, a target selection module and a tracking status feedback module are added to detect all targets and continuously track only a target of interest, that is, to determine whether a certain object in the current frame is the same as the tracked object in the previous frame.

[0162] method This invention Full recognition and tracking Single target tracking Average processing latency per frame (ms) 50 100 30 Tracking status feedback have none none Tracking the lost recovery window (frame) 60 10 0 Tracking trigger methods Manual / Automatic Manual / Automatic Manual

[0163] Table 1. Comparison of characteristics between the present invention and existing detection and tracking methods

[0164] Table 1 shows a comparison of the characteristics of this invention with other existing target recognition and tracking methods. Tests were conducted on a domestic embedded edge computing platform with an NPU computing power of 2 TOPS, using the YOLO11n model as the target detector and the DeepSORT model as the tracker. Conventional multi-target recognition and multi-target tracking schemes have an average processing time of 100 milliseconds per frame, while the multi-target detection and single-target tracking scheme proposed in this invention has an average processing time of 50 milliseconds per frame. This represents a reduction of approximately 50% in latency compared to traditional full-scale tracking schemes, effectively solving the real-time problem of target recognition and tracking on embedded platforms with limited computing power. It also offers greater flexibility in parameter settings and target initialization, and the single-target tracking results are more adaptable to downstream tasks such as servo tracking.

[0165] Compared with typical patents, this invention has advantages. Compared with the full feature extraction of MF-DeepSORT (CN114723786B), this invention can reduce the feature calculation by more than 70% through the target selection module; compared with the dependence of Transformer tracking (CN118247314B) on high-performance GPUs, this invention is optimized for embedded chips, reducing computing resources by about 40%; and compared with the static background limitations of monocular semantic contour tracking (CN202410592006.6), this invention combines motion prediction and appearance features, and can better adapt to complex dynamic environments.

[0166] Regarding target retrieval capabilities, this invention incorporates a state feedback module. When a tracked target is lost due to occlusion or other reasons, it enters a lost target search state, supporting up to 60 frames of lost target search, exceeding the typically less than 10 frames of existing patents, significantly improving the target retrieval rate. Simultaneously, this invention features specific optimizations for edge computing environments, including using squared distance calculations instead of traditional Euclidean distance and implementing dynamic memory management. This enables real-time processing capabilities of 1080p@25fps on an embedded platform with an NPU computing power of 2 TOPS.

[0167] The implementation results of the present invention are as follows: Figure 4 As shown, the top left corner of each frame indicates the frame number, and the bottom left corner indicates the current tracking status. Frame 3 ( Figure 4 At frame A, the target was not locked; by frame 8... Figure 4 B) The algorithm automatically selects the target closest to the center of the frame for tracking and locking until the 380th frame ( Figure 4 C) The target remains locked until frame 403. Figure 4 At time D, the target is partially occluded, but the algorithm still locks onto the target and in frame 417 ( Figure 4 At frame E, the target is completely occluded, and the target search state begins. In this state, all identified targets are sent to the tracker for analysis. (Frame 448) Figure 4 When F) the target reappears, the previously locked target is retrieved and relocked, thus completing the state loop.

[0168] The engineering necessity of this invention's multi-mode design is mainly reflected in three aspects: First, it effectively solves the inherent defects of traditional solutions, including the inability of single-target trackers to handle target loss and the resource waste caused by full multi-target tracking; second, it achieves intelligent tracking management through three-state closed-loop control, with the locking mode ensuring continuous tracking of core targets, the search mode handling target loss anomalies, and the unlocked mode providing a flexible initialization interface. Compared to traditional fixed-mode trackers, this design has significant advantages: it implements a scene-adaptive tracking strategy, automatically switching working modes based on target visibility; it provides personalized interactive design, supporting multiple locking methods; and it establishes a configurable parameter system, allowing key parameters such as locking duration and target loss search window length to be dynamically adjusted according to actual needs. This intelligent multi-mode design not only solves the adaptability problem of traditional solutions in complex scenarios but also significantly improves the system's practicality and reliability through refined state management, enabling it to better meet diverse application needs, especially achieving the best balance between performance and energy efficiency on edge devices with limited computing resources.

[0169] This invention boasts core innovations and technological breakthroughs: First, it creatively proposes a selective tracking paradigm, breaking through the traditional full-scale tracking technical framework; second, it designs a long-term retrieval mechanism, using a 60-frame search window to solve the occlusion recovery problem; third, it achieves embedded-specific optimizations, including innovations in the selective tracking mechanism, computational methods, and memory management, significantly reducing computational overhead compared to traditional solutions; finally, it adopts a universal modular design, compatible with various detectors and trackers, adapting to different application scenarios simply by configuring parameters. These innovations enable this invention to significantly outperform existing technologies in terms of computational efficiency, tracking robustness, flexibility, and embedded adaptability, successfully filling the technological gap in the field of efficient selective target tracking for edge devices and providing a superior technical solution for practical applications such as UAV monitoring and intelligent transportation.

[0170] This invention features a collaborative working mechanism with three states: target lock, unlock, and search. Its main application scenarios include: in long-term, specific target monitoring scenarios (such as security monitoring and wildlife observation), the lock mode ensures continuous and stable tracking of a specific target, effectively preventing tracking loss due to interference. When the target is briefly obscured, the system automatically memorizes the target's features and enters a search state, quickly resuming tracking when the target reappears; in dynamic target selection scenarios (such as drone tracking and intelligent follow-up photography), it can automatically select targets for tracking without requiring a host computer or other input devices; in resource-constrained embedded scenarios, single-target tracking in the lock state significantly reduces computational load, and dynamically allocates computing resources to achieve an intelligent balance between accuracy and energy consumption.

[0171] Finally, it should be noted that the concept, specific structure, and technical effects of the present invention have been clearly and completely described above in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections and connection relationships mentioned herein do not simply refer to direct connection of components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

Claims

1. An embedded multi-target detection and single-target tracking method, characterized in that: include: S1, the target detector detects all targets in the acquired image to obtain the detection box; S2, uses a tracker to number and continuously track targets within the detection frame; S3, the detected detection box and corresponding frame image are sent to the target selection module for splitting and filtering; S4, based on the target selection module, the selected tracking result is input into the tracking feedback module. This module determines in real time whether the tracking result of the current frame matches the tracking result of the previous frame, and adjusts the tracking state according to the matching result, and feeds it back to the target selection module. After model conversion and compilation, the S5 algorithm is deployed to an embedded platform to achieve automatic target detection and tracking.

2. The multi-target detection and single-target tracking method according to claim 1, characterized in that: In step S3, the target selection module performs a detection box splitting process based on the current tracking state, including: The target selection module counts the number of target boxes detected by the detector. If the number of target boxes received by the target selection module is 0, that is, no target is detected in the current frame, the tracking of the current frame will end directly and the detection result of the next frame will be received. If the number of target boxes is not 0, the target box selection stage will be entered, and the tracking will be split according to the current tracking status.

3. The multi-target detection and single-target tracking method according to claim 2, characterized in that: In step S3, the target selection module performs a filtering process for detection boxes based on the current tracking state, including: S301, If ​​the current system state is target locked, traverse all detected target boxes in the current frame, using the center pixel position of the tracked target box in the previous frame as the origin, and calculate the distance between the center of the detected target box and the center of the tracked target box. The sorting is performed, and the distance calculation adopts the square form of the L2 norm of the two-dimensional plane to avoid square root operation and improve real-time performance. This process is expressed as equation (1): ； in, For the traversal of the current frame, the first... The center coordinates of the detected target bounding box For the first The x-coordinate of the center of the detection target bounding box For the first The ordinate of the center of each detection target bounding box. Use the reference coordinates (the target bounding box or the center of the image in the previous frame); This refers to the x-coordinate of the center of the tracking frame in the previous frame or the x-coordinate of the center of the image. The vertical coordinate of the center of the tracking box in the previous frame or the vertical coordinate of the center of the screen is used. Finally, the detection target box that is closest to the tracked object is input into the target tracker. S302, if the current tracking status is target not locked, it means that the current tracking object has not been initialized yet. By default, the detection target box with the shortest Euclidean distance from the center of the screen is selected as the input target tracker. Alternatively, a detection target box can be manually specified as the initial tracking target box and input into the target tracker. S303, If the current tracking state is target loss search, then all received target bounding boxes are input into the target tracker to expand the search range to cope with the reappearance of the target after a brief occlusion. Tracking state diversion strategy. Represented as equation (2): ； in, This represents the set of all detected targets in the current frame. This describes how to select the appropriate target from among these targets based on the three different tracking states [Locked, Unlocked, Searching] to enter the tracker. Indicates the first in the current frame Detection box The center coordinates, Indicates the target box that was locked in the previous frame. The center coordinates, This indicates the center coordinates of the video frame. These coordinates are constant values ​​and are fixed based on the size of the video frame.

4. The multi-target detection and single-target tracking method according to claim 1 or 3, characterized in that: In step S4, the tracking feedback module performs a discrimination process based on the current tracking status, including: S401, When tracking starts, the target is in an unlocked state by default. In this state, the tracking feedback module will determine whether the tracked object has been initialized. If it has not been initialized, the new tracked object will be initialized by the ID and position information of a tracking box output by the tracker. The initialization state is determined by equation (3): ； InitStatus represents the initialization status; it is set to True if the target has been initialized, and False if it has not been initialized. This indicates the initialized target information, including the target bounding box location, category, and target ID. This indicates that the value is not initialized. S402, If the tracking object has been initialized, determine whether the current tracking frame ID matches the initialized tracking object ID; if the IDs do not match, use the latest tracking frame to reinitialize the tracking object ID and position information. The target update rule in the unlocked state is expressed by equation (4): ； in, A unique identifier representing an initialized target. Indicates candidate target information for the current frame. This represents the tracking ID of the candidate target in the current frame; if the ID matches, the initial matching frame counter is incremented by one frame and the position information of the tracked object is updated, as shown in equation (5): ； in, This represents the number of consecutive matching frames at frame t. This indicates the number of consecutive matching frames at frame t-1. Once a certain threshold is reached, a state transition will be triggered. S403, determine whether the number of matching frames has reached the set standard. If the number of matching frames reaches the standard, enter the target locking state. If it does not reach the standard, end the tracking feedback of the current frame and continue to judge the next frame. This process is represented by equation (6): ； in, Indicates the tracking status for the next frame. This represents the minimum number of consecutive matching frames required to enter the locked state.

5. The multi-target detection and single-target tracking method according to claim 4, characterized in that: In step S4, the tracking feedback module performs a discrimination process based on the current tracking state, which also includes: S404, If the system enters the target locking state, it determines whether the latest tracking target ID is consistent with the locked target ID; if they are consistent, it means that the tracking target in the current frame is the same as the tracking target in the previous frame, and the locked target position information can be updated for the target selection module to refer to when selecting the target box in the next frame. Then the tracking result is output. This process is represented by equation (7): ； in, This represents the target location information in frame t. This represents the target ID stored in the locked state; if the current target ID is inconsistent, it means that the target is lost, and the system will unlock the target and enter the lost target search state. The overall state transition process is represented by equation (8): ； S405, In the search state, the tracking feedback module will traverse all the tracking targets output by the tracker and determine whether the latest target IDs are consistent with the tracking object IDs before the target was lost. The process is represented by equation (9): ； in, Indicates whether to retrieve the lost target during the search process. This indicates the total number of tracked targets in the current frame. This represents the ID of the i-th tracked target in frame t; this mechanism enables rapid recovery when the target reappears after occlusion, avoiding tracking drift caused by reinitialization; S406, During the search process, if a target with the same ID as the target locked before the target was lost exists, it means that the lost target has been found, the system will re-enter the locking state, the total number of lost frames will be set to zero, the locked target information will be updated, and the target will continue to be tracked; if all targets are not matched after traversal, it means that no tracked target has been found in the current frame, the search state will continue, and the lost frame counter will be incremented by one frame. This process is shown in equation (10): ； in, This represents the cumulative number of frames lost by the tracked target at frame t in search mode; This represents the number of frames the tracked target lost in frame t-1; S407, In the search state, after each frame search is completed, it is determined whether the total number of lost frames has reached the target lost frame limit. If it has not reached the limit, the current frame tracking feedback ends. If it has reached the limit, the previously tracked object is discarded and the target is re-entered into the target unlocked state. The search state transition process is represented by equation (11): ； in, This parameter sets a pre-defined upper limit on the number of frames a target can be lost in search mode. It ensures that the system automatically resets when a target is lost for an extended period, preventing prolonged ineffective searches from consuming computing resources.

6. The multi-target detection and single-target tracking method according to claim 1, characterized in that: The target detector is any one of YOLO, SSD, or DETR; The tracker can be either DeepSORT or ByteTrack.

7. The multi-target detection and single-target tracking method according to claim 1, characterized in that: This method is applied on an embedded edge computing platform with an NPU computing power of 2 TOPS, using the YOLO11 model as the target detector and the DeepSORT model as the tracker, with an average processing time of 50 milliseconds per frame.

8. A multi-target detection and tracking system for implementing the multi-target detection and single-target tracking method according to any one of claims 1-7, characterized in that: The system includes: An object detector is used to detect multiple objects in an image and outputs the location and category information of the detection boxes; A tracker is used to number the input detection boxes and associate the same target in different frames using these numbers; The target selection module is used to manually select or automatically select specific detection boxes to input into the tracker based on different states; The tracking feedback module is used to compare and analyze the current tracking results, tracking status, and historical tracking information to determine and transfer the tracking status. If the system is in a target lock state, this module is also responsible for updating the location information of the locked target and feeding it back to the target selection module to provide a reference for target selection in the next frame.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the multi-target detection and single-target tracking method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the multi-target detection and single-target tracking method according to any one of claims 1 to 7 by executing the executable instructions.

Images