A multi-target real-time tracking method based on deep learning

Abstract

This invention discloses a multi-target real-time tracking method based on deep learning, comprising: Step 1, acquiring a real-time image sequence through a video acquisition device and standardizing the image sequence; Step 2, inputting the standardized image sequence into a neural network model to output target bounding boxes containing human features; Step 3, inputting the target bounding boxes into a tracker, wherein the tracker calculates the feature vector of each human based on the OSNet appearance feature extraction network, and uses the Hungarian algorithm to solve for the optimal association, assigning a unique identifier to each tracked human, and forming and updating a dynamic tracking list of multiple targets.

Description

[0001] This case pertains to a method and apparatus for controlling a robotic dog to track target objects using gestures in multi-person scenarios, and is a divisional application of application number 2025110760786. Technical Field

[0002] This invention relates to the fields of computer vision and robot control technology, and in particular to a multi-target real-time tracking method based on deep learning. Background Technology

[0003] With the development of intelligent robot dog technology, vision-based target tracking technology is increasingly widely used in service, security, and collaboration fields, especially in robot dogs' need to track specific targets in multi-person scenarios. However, existing robot dog target tracking technologies still have many technical shortcomings in practical applications, making it difficult to meet the tracking requirements of high precision, high stability, and high interactivity, as follows: (1) Insufficient accuracy of target recognition and tracking in multi-person scenes In existing technologies, traditional object detection algorithms are poorly adapted to complex scenes with multiple people, and it is difficult to accurately output bounding boxes containing human features. At the same time, multi-object tracking mechanisms lack continuous labeling and identity management for each object, which can easily lead to object confusion, tracking loss, or identity confusion. They cannot form a stable dynamic tracking list for multiple people, making it difficult for the robot dog to lock onto specific targets.

[0004] (2) Lack of natural and intuitive human-computer interaction and goal setting methods Existing robot dog tracking systems mostly rely on buttons, touch screens, or voice commands to select targets and control actions. The interaction methods are cumbersome and unnatural. In multi-person scenarios, it is difficult to quickly and accurately specify the target to be tracked, and it is also impossible to start or stop the tracking task through intuitive actions.

[0005] (3) The accuracy of the calculation of the target's relative spatial parameters is low. In existing technologies, the calculation of the horizontal angle and distance of the target relative to the robot dog mostly relies on simple geometric estimation, without combining the internal parameters of the video acquisition device for accurate modeling. This results in large errors in the angle and distance parameters, which cannot provide a reliable spatial basis for the robot dog's motion control. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies in target tracking in multi-person scenes, such as low accuracy of target recognition and tracking, unnatural interaction methods, and insufficient accuracy of spatial parameter calculation, and to provide a method and device for gesture-controlled robot dogs to track target objects in multi-person scenes.

[0007] The objective of this invention is achieved through the following technical solution: A method for controlling a robot dog to track a target object using gestures in a multi-person scenario includes the following steps: Step 1: Acquire real-time image sequences using video capture equipment and perform standardization processing on the image sequences; Step 2: Input the standardized image sequence into the neural network model, output target bounding boxes containing human features, and then use a tracker to continuously track the people in multiple target bounding boxes and assign unique identification identifiers to form a dynamic tracking list of multiple human targets. Step 3: Perform gesture recognition on each person based on the dynamic tracking list, and determine the target object that the robot dog is tracking and the action that the robot dog is to perform based on the recognized gestures. Step 4: Based on the internal parameters of the video acquisition device and the coordinates of the target bounding box and the center coordinates, calculate the horizontal angle and distance between the person and the robot dog; Step 5: Based on the horizontal angle and distance, the robot dog's linear and angular movements are controlled by distance control PID and angle control PID, converting the spatial parameters into specific motion commands to drive the robot dog to perform target object tracking actions.

[0008] Preferably, step 1 involves standardizing the image sequence, including: The image sequence is converted to a different color space, from YUYV to BGR format. The BGR format image is resized using bilinear interpolation to scale it to the input size required by the neural network model. The scaled image is numerically normalized, converting pixel values ​​from an integer range of 0-255 to a floating-point range of 0-1.

[0009] Preferably, the neural network model is the YOLOv5s neural network model, which includes a backbone feature extraction network, a path aggregation feature fusion network, and a multi-scale detection head. The tracker described is the DeepSORT tracker, which uses the OSNet appearance feature extraction network to calculate the feature vector of each person; the Hungarian algorithm is used to solve for the optimal association and assign a unique identifier to each tracked person.

[0010] As a preferred option, a detection result file interface based on JSON format is also constructed to enable data transmission between the detection process and the control process. This JSON-based cross-process communication interface ensures the consistency and real-time performance of data interaction between the detection and control processes, as well as the efficient transmission of motion commands, thereby improving the system's response speed.

[0011] As a preferred embodiment, when the robot dog performs target object tracking, a multi-stage target loss handling mechanism is established for target loss recovery and motion maintenance, specifically: In Phase 1, when the target is detected to be lost, the robot dog enters motion-holding mode, stops moving forward, and maintains the last effective angle of motion. Phase Two: When the target loss time exceeds the preset timeout threshold, the execution status is reset.

[0012] By maintaining motion and resetting the state, the tracking system can actively search for and resume tracking when the target is temporarily lost, and reset the state when the target is lost for a long time, thus improving the robustness and anti-interference ability of the tracking system.

[0013] Preferably, the robot dog is communicated in real time via the UDP protocol. When sending motion control commands, the calculated motion command parameters are encoded in double-precision floating-point format and transmitted to the robot dog's execution system.

[0014] A device for controlling a robot dog to track a target object using gestures in a multi-person scenario, and a method for controlling a robot dog to track a target object using gestures in a multi-person scenario, comprising: The video acquisition and preprocessing module is used to acquire real-time image sequences through video acquisition equipment and to perform standardization processing on the image sequences; The target detection and tracking module is used to input the standardized image sequence into the neural network model, output target bounding boxes containing human features, and then continuously track the people in multiple target bounding boxes and assign unique identifications to them, forming a dynamic tracking list of multiple human targets. The gesture recognition and target action determination module is used to perform gesture recognition on each person based on a dynamic tracking list, and determine the target object that the robot dog is tracking and the action that the robot dog is to perform based on the recognized gesture. The angle and distance calculation module is used to calculate the horizontal angle and distance between the person and the robot dog based on the internal parameters of the video acquisition device and the coordinates of the target bounding box and the center coordinates. The motion control module is used to control the linear and angular movements of the robot dog based on horizontal angle and distance, through distance control PID and angle control PID, converting spatial parameters into specific motion commands to drive the robot dog to perform target object tracking actions.

[0015] A storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, steps are taken to implement a method for controlling a robot dog to track a target object using gestures in a multi-person scenario.

[0016] The beneficial effects of this invention are: 1. By organically combining video acquisition and preprocessing, target detection and tracking with neural networks and trackers, gesture recognition based on dynamic tracking lists, spatial parameter calculation, and dual PID motion control, the system systematically solves the problems of low target recognition and tracking accuracy, unnatural interaction methods, and insufficient spatial parameter calculation accuracy in multi-person scenarios of existing technologies. It realizes the robot dog's accurate and stable tracking of specific targets, and significantly improves the reliability of tracking tasks in complex multi-person scenarios.

[0017] 2. By combining the YOLOv5s neural network model with the DeepSORT tracker, and integrating appearance feature extraction and the Hungarian algorithm, the system can accurately output the bounding box of the target person and assign a unique identifier, effectively avoiding target confusion and tracking loss in multi-person scenes, and significantly improving the accuracy and continuity of multi-target tracking.

[0018] 3. The YOLOv5s neural network model and DeepSORT tracker rely on the NPU acceleration capability and are directly deployed locally on the development board to achieve lightweight and efficient operation.

[0019] 4. The relative horizontal angle and distance of the target are calculated based on the internal parameters of the video acquisition device, which significantly improves the accuracy of spatial parameter calculation, provides a reliable basis for the motion control of the robot dog, and reduces the offset and jitter during the tracking process. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0022] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0023] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0025] Example: Methods for using gestures to control a robot dog to track target objects in multi-person scenarios, such as... Figure 1 As shown, it includes the following steps: Step 1: Acquire real-time image sequences using video capture equipment and perform standardization processing on the image sequences; Step 2: Input the standardized image sequence into the neural network model, output target bounding boxes containing human features, and then use a tracker to continuously track the people in multiple target bounding boxes and assign unique identification identifiers to form a dynamic tracking list of multiple human targets. Step 3: Perform gesture recognition on each person based on the dynamic tracking list, and determine the target object that the robot dog is tracking and the action that the robot dog is to perform based on the recognized gestures. Step 4: Based on the internal parameters of the video acquisition device and the coordinates of the target bounding box and the center coordinates, calculate the horizontal angle and distance between the person and the robot dog; Step 5: Based on the horizontal angle and distance, the robot dog's linear and angular movements are controlled by distance control PID and angle control PID, converting the spatial parameters into specific motion commands to drive the robot dog to perform target object tracking actions.

[0026] In step 1, the device parameters are set as follows: frame rate f equals 30 frames per second, resolution W multiplied by H equals 640 pixels multiplied by 480 pixels, and format is YUYV. The obtained original image sequence is denoted as I(x,y,t), where x and y represent the pixel coordinates in the image, and t represents the timestamp. The image sequence is then standardized, including: The image sequence is converted to a different color space, from YUYV to BGR format. BGR image(u,v) = color space conversion function(YUYV image(u,v)) Where u and v represent the pixel coordinates of the converted image, and the color space conversion adopts the conversion matrix of the ITU-R BT.601 international standard.

[0027] The BGR format image is resized using bilinear interpolation to scale it to the input size of 640×640 required by the neural network model. Adjusted image (u,v) = Bilinear interpolation scaling (BGR image, 640, 640) The scaled image is numerically normalized, converting pixel values ​​from an integer range of 0-255 to a floating-point range of 0-1.

[0028] Normalized image (u,v,c) = Adjusted image (u,v,c) ÷ 255.0 Where 'c' represents the color channel, which includes three channels: red (R), green (G), and blue (B).

[0029] The neural network model mentioned is the YOLOv5s neural network model, which includes a backbone feature extraction network, a path aggregation feature fusion network, and a multi-scale detection head. The tracker described is the DeepSORT tracker, which uses the OSNet appearance feature extraction network to calculate the feature vector of each person; the Hungarian algorithm is used to solve for the optimal association and assign a unique identifier to each tracked person.

[0030] Specifically, the formula for calculating the feature extraction of the l-th layer of the backbone network is: Layer l features = activation function (batch normalization (convolution operation (layer l-1 features))) Where l represents the number of network layers, the activation function is SiLU, batch normalization is used to stabilize the training process, and convolution operation is used for feature extraction.

[0031] The detection head outputs the bounding box information for each candidate object: Bounding box = [center x, center y, width, height, confidence score, class probability] The test results are screened for quality, and test frames that meet the following conditions are retained: Confidence level × maximum class probability ≥ confidence threshold The confidence threshold is set to 0.7 to ensure the reliability of the detection results.

[0032] Apply the non-maximum suppression algorithm to remove duplicate detections, and calculate the intersection-union ratio (CUC) of two detection boxes: Intersection-to-union ratio = Area of ​​overlapping region ÷ Area of ​​total covered region When the cross-union ratio is greater than 0.45, the detection boxes with low confidence are deleted to avoid duplicate detection of the same target.

[0033] The person detection results are input into the DeepSORT tracker for multi-object tracking. DeepSORT uses the OSNet appearance feature extraction network to calculate the feature vector for each object. Feature vector = OSNet network (cropped target image) A data association strategy combining appearance features and motion models is adopted: Association cost matrix = λ × appearance similarity cost + (1-λ) × motion prediction cost Where λ equals 0.7, representing the weight of appearance features in the association decision. The optimal association is solved using the Hungarian algorithm, assigning a unique tracking identifier to each tracked target.

[0034] In this embodiment, two categories of control gestures are defined: Gesture category = {fist gesture (number 5), stop gesture (number 1)} The fist gesture is used to start target tracking, while the stop gesture is used to terminate the current tracking task.

[0035] Establish a mechanism for determining the validity of gestures to ensure the accuracy of gesture control: Valid gesture = (confidence ≥ 0.75) and (duration ≥ 1.0 second) The confidence level of 0.75 is the minimum confidence requirement for gesture recognition, and the duration of 1.0 second is used to avoid erroneous triggering caused by momentary false detection.

[0036] Achieve spatial correlation matching between gestures and human targets. Calculate the Euclidean distance between the center points of the gesture detection box and the human detection box:

[0037] When the spatial distance is less than or equal to the maximum association distance (set to 200 pixels), establish an association between the gesture and the person.

[0038] Establish an intelligent target selection strategy; when a valid fist gesture is detected: Selected target = the person closest to the gesture location.

[0039] A JSON-based interface for detecting results files was also built to facilitate data transmission between the detection and control processes. This JSON-based cross-process communication interface ensures consistency and real-time performance of data interaction between the detection and control processes, as well as efficient transmission of motion commands, thereby improving the system's response speed.

[0040] Specifically, the detection data structure = { Timestamp: Current timestamp in milliseconds. Frame number: Current frame sequence number, Detection results list: [Target object 1, Target object 2, ...], Target marker information: Current tracking status } Each target object contains the following information: tracking identity, bounding box coordinates, confidence level, category number, whether it is the current target, and target center coordinates.

[0041] Establish a target identity control file interface, supporting two working modes: Control mode = { Automatic mode: "auto" - Automatically selects targets based on gestures. Manual mode: Specific identification value - tracking the target with the specified number} Atomic file operations are used to ensure the consistency and reliability of data transmission: the data update process is completed in the steps of temporary file writing, atomic renaming operation, and data file update. In step 4, the intrinsic parameter matrix obtained by camera calibration is used to perform accurate angle and distance calculations. The standard form of the camera intrinsic parameter matrix K is: K = [focal length fx0 principal point cx][0 focal length fy principal point cy]

[001] where fx and fy represent the focal length parameters in the horizontal and vertical directions, respectively, and cx and cy represent the coordinates of the principal point of the image.

[0042] Calculate the horizontal angle of the target relative to the camera: Horizontal angle = arctangent function ((target center x-coordinate - principal point cx-coordinate) ÷ horizontal focal length fx) Target distance is estimated based on the size of the target detection box: Estimated distance = (Actual target width × Horizontal focal length fx) ÷ Detection box width, where the actual target width is set to 0.5 meters, representing the average shoulder width of an adult. An intelligent correction algorithm is implemented for truncated targets at the image edge. The edge region is defined as the area within 50 pixels of the image boundary: Truncation determination = (Target x-coordinate < 50) or (Target x-coordinate + Target width > Image width - 50) When a target is determined to be truncated and the detection box width is less than 120 pixels and the absolute angle value is greater than 15 degrees, width correction is performed: Correction coefficient = 1.0 + (absolute angle value - 15 degrees) × 0.02. Corrected width = original detection box width × minimum value (correction coefficient, 1.5). The maximum correction coefficient is limited to 1.5 to avoid overcorrection. A Kalman filter is applied to smooth the distance and angle estimates: Predicted state = Kalman prediction (previous state) Corrected state = Kalman correction (predicted state, current measurement value). When the robot dog performs target tracking, a multi-stage target loss handling mechanism is established for target loss recovery and motion maintenance. Specifically: Stage 1, when target loss is detected, the robot dog enters motion maintenance mode, stops moving forward, and maintains the last effective angle of movement. The design idea of ​​this strategy is to stop moving forward to avoid collisions when the robot dog loses target guidance, but maintain turning motion in order to rediscover the target.

[0043] Phase Two: When the target loss time exceeds the preset timeout threshold, a state reset is performed, resetting the gesture detection state, clearing the tracking history, and returning to the waiting gesture activation mode.

[0044] The four operating states of the system are as follows: System state = { Waiting for gesture activation: After system startup, it waits for user gesture activation. Normal tracking status: Target successfully locked and tracking control initiated. Intelligent state retention: the relocation and search phase after target loss. Standby reset state: The system reset phase after a long period of data loss. }

[0045] By maintaining motion and resetting the state, the tracking system can actively search for and resume tracking when the target is temporarily lost, and reset the state when the target is lost for a long time, thus improving the robustness and anti-interference ability of the tracking system.

[0046] In step 5, the distance control PID algorithm is as follows: Distance error = Current distance - Target distance Distance PID output = Kp distance × distance error + Ki distance × distance error integral + Kd distance × distance error derivative Wherein Kp distance, Ki distance, and Kd distance are the proportional, integral, and derivative coefficients for distance control, respectively, and the set values ​​are: Kp distance = 0.8, Ki distance = 0.15, and Kd distance = 0.05.

[0047] The specific PID algorithm for angle control is as follows: Angle PID output = Kp (angle × angle error) + Ki (angle × angle error integral) + Kd (angle × angle error derivative) The angle control parameters are set as follows: Kp angle = 1.5, Ki angle = 0.03, Kd angle = 0.20.

[0048] Implement motion parameter limitations to ensure the safety of the robot dog's movements: Final linear speed = Limiting function (distance to PID output, maximum linear speed) Final angular velocity = Limiting function (Angle PID output, Maximum angular velocity) The maximum linear velocity is set to 0.3 meters per second, and the maximum angular velocity is set to 1.0 radians per second.

[0049] Increasing the limit on the rate of change of velocity improves the smoothness of motion: Smooth linear velocity = Rate of change limit (Target linear velocity, Current linear velocity, Maximum linear acceleration × Time interval) Smooth angular velocity = Rate of change limit (Target angular velocity, Current angular velocity, Maximum angular acceleration × Time interval) The maximum linear acceleration was set to 0.5 m / s², and the maximum angular acceleration was set to 2.0 radians / s².

[0050] In this embodiment, a high-frequency control strategy of 20 Hz is implemented, which is 100% higher than the traditional 10 Hz control frequency. Control cycle = 1 ÷ 20 = 0.05 seconds High-frequency control can significantly improve the smoothness and response speed of robot dog movements, and reduce the impact of control delay on tracking accuracy.

[0051] Simultaneously, a multi-level status monitoring mechanism should be established: Normal tracking status update frequency = 4 Hz Target loss status monitoring frequency = 4 Hz Target identity change detection frequency = 3.33 Hz.

[0052] Real-time communication with the robot dog is achieved via the UDP protocol. When sending motion control commands, the calculated motion command parameters are encoded in double-precision floating-point format and transmitted to the robot dog's execution system. Specifically, real-time communication with the robot dog is conducted using the UDP protocol, with the communication parameters set as follows: robot dog IP address 192.168.1.6, communication port 43893.

[0053] Use the cloud-deep lite standardized UDP control command format: Instruction structure = [Instruction code (4 bytes), Parameter length (4 bytes), Data type (4 bytes), Data content] Define motion control instruction codes: Linear speed control instruction code = 0x0140 Angular velocity control command code = 0x0141 Lateral speed control command code = 0x0145 Autonomous mode switching instruction code = 0x21010C03.

[0054] A device for controlling a robot dog to track a target object using gestures in a multi-person scenario, and a method for controlling a robot dog to track a target object using gestures in a multi-person scenario, comprising: The video acquisition and preprocessing module is used to acquire real-time image sequences through video acquisition equipment and to perform standardization processing on the image sequences; The target detection and tracking module is used to input the standardized image sequence into the neural network model, output target bounding boxes containing human features, and then continuously track the people in multiple target bounding boxes and assign unique identifications to them, forming a dynamic tracking list of multiple human targets. The gesture recognition and target action determination module is used to perform gesture recognition on each person based on a dynamic tracking list, and determine the target object that the robot dog is tracking and the action that the robot dog is to perform based on the recognized gesture. The angle and distance calculation module is used to calculate the horizontal angle and distance between the person and the robot dog based on the internal parameters of the video acquisition device and the coordinates of the target bounding box and the center coordinates. The motion control module is used to control the linear and angular movements of the robot dog based on horizontal angle and distance, through distance control PID and angle control PID, converting spatial parameters into specific motion commands to drive the robot dog to perform target object tracking actions.

[0055] A storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, steps are taken to implement a method for controlling a robot dog to track a target object using gestures in a multi-person scenario.

[0056] This embodiment introduces a control mode that starts tracking with a fist gesture and terminates tracking with a stop gesture, providing a natural and intuitive human-computer interaction interface for the robot dog. Through dual safeguards of 1-second continuous detection and a 0.75 confidence threshold, false triggers are effectively avoided, improving the reliability of user operation. Addressing the technical challenge of inflated distance estimation due to the detection box shrinking when the target moves to the edge of the screen, an angle-based intelligent correction algorithm is proposed. Through geometric relationship modeling and a linear correction strategy, the distance measurement error in the edge region is reduced from 200-300% to within 20-30%.

[0057] The technical solution of this embodiment can be widely applied in the following fields: Intelligent security patrol: Deploy patrol robot dogs in shopping malls, parks and other places, and control them to track suspicious targets via gestures.

[0058] Human-robot collaborative factory: On the production line, workers and robot dogs collaborate, and the robot dogs can track specific workers based on their gestures.

[0059] Service robot dog applications: In service venues such as hotels and hospitals, robot dogs can provide following services to specific customers through gesture recognition.

[0060] Smart home companionship: A family service robot dog can track family members through gesture control and provide personalized services.

[0061] Exhibition and demonstration system: Used as an interactive demonstration system in science and technology exhibitions to showcase advanced robot dog tracking technology.

[0062] The system can run on edge computing platforms such as RK3588, supporting real-time detection and tracking processing at 30 frames per second, meeting the real-time control needs of various mobile robot dogs. Through modular software design, it can be quickly adapted to different types of robot dog platforms and application scenarios.

[0063] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0064] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A deep learning-based multi-target real-time tracking method, characterized in that, Includes the following steps: Step 1: Acquire real-time image sequences using video capture equipment and perform standardization processing on the image sequences; Step 2: Input the standardized image sequence into the neural network model and output the target bounding box containing human features; Step 3: Input the target bounding box into the tracker. The tracker calculates the feature vector of each person based on the OSNet appearance feature extraction network, and uses the Hungarian algorithm to solve the optimal association. It assigns a unique identifier to each tracked person, forming and updating a dynamic tracking list of multiple targets. 2.The deep learning based multi-target real-time tracking method according to claim 1, wherein, The neural network model described is the YOLOv5s model, which includes a backbone feature extraction network, a path aggregation feature fusion network, and a multi-scale detection head. 3.The deep learning based multi-target real-time tracking method according to claim 1, wherein, The tracker is a DeepSORT tracker.

4. The multi-target real-time tracking method based on deep learning according to claim 1 or 3, characterized in that, In step S3, a data association strategy combining appearance features and motion is employed, as follows: Association cost matrix = λ × appearance similarity cost + (1-λ) × motion prediction cost Where λ is the weight of appearance features in association decision.

5. The multi-target real-time tracking method based on deep learning according to claim 2, characterized in that, The detection head outputs bounding box information for each candidate target: Bounding box = [center x, center y, width, height, confidence score, class probability] The test results are screened for quality, and test frames that meet the following conditions are retained: Confidence level × Maximum class probability ≥ Confidence threshold The non-maximum suppression algorithm is applied to remove duplicate detections. The cross-union ratio (CUP) of two detection boxes is calculated. When the CUP is greater than 0.45, the detection box with the lower confidence is deleted.

6. The multi-target real-time tracking method based on deep learning according to claim 1, characterized in that, It also includes building a detection result file interface based on JSON format to realize data transmission between the detection process and subsequent application processes; the detection result data includes timestamp, frame number, tracking identity identifier, bounding box coordinates, confidence level, category number, whether it is the current target, target center coordinates, and target label information.

7. The multi-target real-time tracking method based on deep learning according to claim 1, characterized in that, It also includes a multi-stage target loss handling mechanism, specifically: In Phase 1, when the target is detected to be lost, the robot dog enters motion-holding mode, stops moving forward, and maintains the last effective angle of motion. Phase Two: When the target loss time exceeds the preset timeout threshold, a state reset is performed, resetting the gesture detection state, clearing the tracking history, and returning to the waiting gesture activation mode.

8. The multi-target real-time tracking method based on deep learning according to claim 1, characterized in that, The standardized preprocessing includes color space conversion, image size adjustment, and pixel value normalization.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the deep learning-based multi-target real-time tracking method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based multi-target real-time tracking method as described in any one of claims 1 to 8.

Images