Target tracking method fusing visual perception and motion control

Abstract

The invention discloses a target tracking method fusing visual perception and motion control. The method comprises the following steps: improving a YOLOv8 model and pre-training the improved YOLOv8 model; s2, preprocessing an image acquired by a camera, inputting the preprocessed image into the YOLOv8 model trained in the step S1, and outputting a target bounding box and a target center pixel coordinate; target center pixel coordinates are transmitted to a development board of a pre-burning algorithm program through a serial port, the development board is designed according to a stepping motor mathematical model, sine acceleration and deceleration serve as the basis, PID control, sliding mode control and fuzzy control algorithms are fused, and code burning is generated after in-loop simulation optimization; and the development board drives a stepping motor to adjust the posture of the holder according to the target center pixel coordinate control instruction, so that the target stops moving after being kept in the view center. The problems that a traditional YOLOv8 model is large in target recognition error under the complex background, PID control is prone to oscillation under the nonlinear working condition, buffeting exists in sliding mode control, and response lags when fuzzy control is independently used are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent control and computer vision, and particularly relates to a target tracking method that integrates visual perception and motion control. Background Art

[0002] Target tracking systems face numerous challenges in drone reconnaissance, intelligent control of industrial inspection, and computer vision technology. Traditional target tracking methods rely on single sensors or simple algorithms, making them difficult to address in complex scenes, such as target occlusion, varying lighting conditions, and multi-target interference. While YOLOv8-based deep learning target detection algorithms are efficient and real-time, native models lack the ability to extract large target features and fuse multi-scale features. This limits detection accuracy in complex environments, and ID continuity and occlusion recovery capabilities in multi-target tracking require improvement.

[0003] Regarding the stepper motor control used in two-degree-of-freedom gimbals, traditional PID control is prone to oscillation and steady-state errors under nonlinear conditions. Sliding mode control, while robust, suffers from chattering. Fuzzy control alone suffers from insufficient response speed. A single control strategy struggles to balance dynamic response, interference rejection, and steady-state accuracy. Regarding the coordination of visual perception and motion control, the nonlinear mapping of image coordinates to physical motion can easily lead to tracking errors and lacks a real-time dynamic calibration mechanism. The computing power limitations of embedded platforms also pose challenges to the deployment of deep learning models. Summary of the Invention

[0004] To overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a target tracking method that integrates visual perception and motion control, so as to solve the problems that the traditional YOLOv8 model has large target recognition errors in complex backgrounds, PID control is prone to oscillation under nonlinear conditions, sliding mode control has chattering, and fuzzy control has response lag when used alone.

[0005] The technical solution adopted in the present invention is as follows:

[0006] A target tracking method integrating visual perception and motion control, comprising the steps of:

[0007] S1. Improve the YOLOv8 model and pre-train the improved YOLOv8 model;

[0008] The improved YOLOv8 model specifically introduces the P6 layer in the backbone network, extracts global features of large objects through 3×3 convolution downsampling and the C2f module; integrates the ChannelTransformer module in the neck network, and solves the problem of multi-scale feature semantic gaps through a cross-channel attention mechanism; and integrates the ConvFormer module in the detection head, combining the Transformer self-attention mechanism to enhance the ability to capture long-range dependencies.

[0009] S2, after preprocessing, the image captured by the camera is input into the YOLOv8 model trained in step S1, and the target bounding box and the pixel coordinates of the target center (x i ,y i );

[0010] S3, the target bounding box and target center pixel coordinates (x i ,y i ) is transferred to the development board of the pre-burned algorithm program, where x i ,y i Respectively represent the horizontal and vertical coordinates of the target center in the image coordinate system; the development board is designed based on the mathematical model of the stepper motor and is based on sinusoidal acceleration and deceleration. It integrates PID control, sliding mode control and fuzzy control algorithms, and generates code for burning after in-loop simulation optimization;

[0011] S4, the development board calculates the target boundary box and the target center pixel coordinates (x i ,y i ) generates control instructions to drive the stepper motor to adjust the gimbal posture so that the target remains in the center of the field of view and then stops moving.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] This invention uses a stepper motor drive and establishes its transfer function and state-space mathematical model. It also improves the YOLOv8 algorithm, introducing the P6 layer, ChannelTransformer, and ConvFormer modules to enhance large target feature extraction and multi-scale feature fusion capabilities. DeepSORT is combined with this method to achieve multi-target tracking and deploy it to the camera. Fuzzy PID control is used in the stable range, and sliding mode control is switched to in the strong interference range. A sinusoidal acceleration and deceleration algorithm is combined to optimize the motor's dynamic response. Visual coordinate data is transmitted to the development board via a serial port, and a visual-to-motion dynamic mapping model is established. Accurate tracking is achieved using encoder feedback and dead-zone thresholds in the stepper motor. Ablation experiments have verified that the improved model's mAP@0.5 improves by 12.8% and mAP@0.95 improves by 6.6%, effectively addressing the problems of insufficient target detection accuracy in complex environments, the conflict between motor control dynamic performance and steady-state accuracy, and visual-to-motion coordination errors.

[0014] In summary, the present invention improves the accuracy and stability of target tracking in complex environments by improving visual algorithms, integrating control strategies and optimizing system collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a process framework diagram of the present invention.

[0016] Figure 2 Flow chart of the method of the present invention.

[0017] Figure 3 This is a flow chart of the stepping motor operation of the present invention.

[0018] Figure 4 This is a flow chart of the PID control algorithm proposed in this invention.

[0019] Figure 5 This is a schematic diagram of the synovial control principle proposed in the present invention.

[0020] Figure 6 This is the fuzzy control principle diagram proposed in this invention.

[0021] Figure 7 This is the principle diagram of the stepper motor control algorithm proposed in this invention.

[0022] Figure 8 This is a flowchart of the improved BackBone algorithm proposed in this invention.

[0023] Figure 9 This is a flowchart of the improved YOLO target tracking system algorithm proposed in this invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of this solution will be clearly and completely described below in conjunction with the drawings in the embodiments of this solution. Obviously, the described embodiments are only part of the embodiments of this solution, not all of the embodiments; based on the embodiments in this solution, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this solution.

[0025] In view of the importance and existing problems of airborne inertial stabilized gimbals in target tracking, the present invention designs a two-degree-of-freedom gimbal structure, constructs a mathematical model of a stepper motor, and derives a sliding mode and fuzzy PID controller. In Simulink, a variety of control algorithms are studied, different acceleration and deceleration methods are compared, a fusion algorithm is proposed, and simulation verification shows that it has high control accuracy and smooth operation. The yolov8 algorithm is selected, and by collecting labeled data sets and training the model using the PyTorch framework, the algorithm is improved to enhance the large target detection capability and perform ablation experiment optimization, and is combined with the DeepSORT algorithm to deploy it to the camera to achieve target tracking. A gimbal tracking experimental platform is built, and experimental tests have proved that the improved system motor motion accuracy is improved. The present invention combines image recognition and stepper motor drive technology, and uses Simulink functions to lower the code writing threshold, providing ideas and theoretical references for research in related fields, and has application potential in many fields.

[0026] A target tracking method that integrates visual perception and motion control is developed around a two-degree-of-freedom pan-tilt target tracking system. First, the hardware is built, the two-degree-of-freedom pan-tilt is assembled, and the stepper motor, sensor, driver, controller, USB camera and other devices are connected and the relevant parameters are configured. Then, the algorithm design and implementation work is carried out. In Simulink, based on the mathematical model of the stepper motor, an algorithm based on sinusoidal acceleration and deceleration is designed, integrating PID, sliding mode and fuzzy control. After in-the-loop simulation optimization, the code is generated and burned into the STM32 development board. At the same time, the YOLOv8 algorithm is selected, and annotated data sets are collected. The model is trained using the PyTorch framework, improved and optimized through ablation experiments, and then combined with the DeepSORT algorithm and deployed to the camera. Then, model training and optimization are carried out, and the performance of the YOLOv8 model is evaluated and adjusted based on multiple indicators. The optimal model is determined through ablation experiments. Finally, the overall operation and testing of the system are carried out. After the camera recognizes the target, the coordinates are sent to the development board, the motor drives the pan-tilt to track the target, and the encoder is used to collect motor data to draw curves for performance analysis.

[0027] The structure of the two-degree-of-freedom platform includes the following main parts:

[0028] Base: The base is the supporting structure of the entire turntable, providing stability and rigidity while bearing the load of the upper components; Pitch rotation axis: Responsible for achieving pitch angle movement, driven by a stepper motor, with the worm gear and worm gear transmitting power; the stepper motor is connected to the rotation axis of the gimbal through a worm gear transmission; the worm gear transmission has the advantages of a large transmission ratio, a compact structure, and good self-locking performance; the large transmission ratio can convert the high-speed rotation of the stepper motor into a low-speed, high-torque rotation of the gimbal, allowing the gimbal to adjust its angle smoothly; the self-locking performance ensures that the gimbal can maintain its current position when it stops moving and will not deviate due to interference from external forces, thereby improving the stability and reliability of the system; Horizontal rotation axis: Located above the pitch axis, it realizes horizontal rotation;

[0029] Drive system: Composed of stepper motors, they directly drive and transmit torque for precise motion control. A 57mm stepper motor is used as the power source, with a step angle of 1.2° and a phase current of 3.5A. The stepper motor has precise position control capabilities and can accurately rotate a certain angle based on the input pulse signal, thereby achieving precise movement of the gimbal. By controlling the rotation of the stepper motor, the pitch and horizontal angles of the gimbal can be precisely adjusted, allowing the visual device to accurately align with the target.

[0030] Sensor system: Consists of feedback elements such as encoders or gyroscopes, used to detect real-time motion status and ensure high-precision closed-loop control of the system; Control algorithm: Uses PID control and other complex integrated control algorithms to adjust the motor based on sensor data to ensure gimbal stability and accurate movement.

[0031] Visual acquisition equipment: The PTZ is equipped with a USB camera for real-time acquisition of visual information of the target. USB cameras offer advantages such as easy installation and stable data transmission. They can quickly and accurately transmit acquired image data to the system for processing. By analyzing the images captured by the camera, information such as the target's position, size, and motion state can be determined, providing a basis for the PTZ's motion control.

[0032] like Figure 2 As shown in the figure, the visual acquisition equipment is mainly used to obtain the image information of the target in real time, providing the data basis for subsequent target detection, recognition and tracking:

[0033] S1. High resolution and wide-angle field of view: Supports 4K resolution (3840×2160 pixels), capable of capturing high-definition images. The wide-angle lens can expand the field of view to meet the needs of target search in complex environments, especially suitable for large-scale monitoring scenarios such as drones or fixed points.

[0034] S2, connected to the embedded system (such as STM32 development board or host computer) through the USB interface, without the need for additional drivers, easy deployment, compatible with MATLAB, Python and other development environments, easy to integrate with the YOLOv8 algorithm;

[0035] Real-time data transmission: supports real-time video streaming transmission with a frame rate of up to 30FPS (frames per second), meeting the real-time requirements of target tracking and ensuring continuous capture of dynamic targets;

[0036] The S3 and USB cameras capture scene images in real time, serving as the input data source for the YOLOv8 algorithm. Through lens optical imaging, they project targets in three-dimensional space into two-dimensional images, providing raw visual information for target detection. For example, in industrial inspection scenarios, they can capture the real-time position of workpieces on the production line; in drone reconnaissance, they can capture dynamic images of ground targets.

[0037] S4. Integrate YOLOv8 target detection with deep learning algorithms: After preprocessing the images captured by the camera (such as resizing to 640×640 pixels), they are input into the improved YOLOv8 model to achieve target detection and positioning;

[0038] 5. The model extracts features of large targets and complex backgrounds through the backbone network (including the P6 layer) and the neck network (ConvFormer module), and outputs the target bounding box and center coordinates (xi, yi);

[0039] S6. Subsequently, DeepSORT performs multi-target tracking: combining the continuous frame images of the camera, using the DeepSORT algorithm to associate the targets in the previous and next frames, predicting the trajectory through Kalman filtering, and matching the appearance features (such as cosine distance) and motion features (such as Mahalanobis distance) through the Hungarian algorithm to solve the problems of target occlusion and ID continuity;

[0040] The specific implementation of the multi-target tracking includes:

[0041] Cascade matching and feature fusion strategy DeepSORT algorithm handles target occlusion through cascade matching mechanism, matching cost matrix combined with Mahalanobis distance d Ma h alanobis and the apparent feature cosine distance d cosine , the formula is: C = 0.9d Ma h alanobis +0.1d cosine ,

[0042] The Mahalanobis distance measures the consistency of the target's motion state, and the cosine distance measures the similarity of the target's appearance features (such as the 128-dimensional feature vector extracted by ResNet50);

[0043] In a test scenario containing 10 dynamic targets, the system achieved a target ID continuity retention rate of more than 95%, an average tracking error of less than 8 pixels, and a processing frame rate of 25FPS, meeting real-time requirements; for targets with occlusion time exceeding 2 seconds, the re-identification success rate reached 83%, significantly better than the traditional DeepSORT algorithm (72%).

[0044] S7, coordinate data output The camera transmits the detected target center pixel coordinates to the STM32 microcontroller in real time through the serial port (such as UART protocol). The coordinate data format is (xi, yi), where (xi, yi), xi, yi represent the horizontal and vertical coordinates of the target center in the image coordinate system;

[0045] S8, this data is used as the input command for the gimbal motion control, driving the stepper motor to adjust the gimbal posture so that the target remains in the center of the field of view;

[0046] S9. Collaboration mechanism with the PTZ system. Definition of the image coordinate system: The center of the camera imaging plane is the origin (x c ,y c ), establish a two-dimensional coordinate system; the target center coordinate (x i ,y i ) Deviation from the origin Δ=x i -x c , Δy=y i -y c Directly reflects the position offset of the target in the field of view;

[0047] S10, drive the gimbal to move according to the deviation value: if Δx>0, the gimbal rotates horizontally to the left, i.e. the horizontal motor rotates in the reverse direction; if Δx<0, the gimbal rotates horizontally to the right, i.e. the horizontal motor rotates in the forward direction;

[0048] S11, if Δy>0, the gimbal pitches downward, i.e. the pitch motor rotates in the reverse direction; if Δy<0, the gimbal pitches upward, i.e. the horizontal motor rotates in the forward direction;

[0049] When the deviation falls within the preset dead zone (e.g., ±20 pixels), the target is considered aligned, the gimbal stops moving, and vibration is suppressed.

[0050] S12, closed-loop feedback optimization: The images collected by the camera are not only used for target detection, but also indirectly feedback the gimbal motion accuracy through the encoder;

[0051] S13. When the gimbal rotates, the camera monitors the target position changes in real time, forming a closed-loop process of "detection → control → re-detection" to correct tracking deviations caused by mechanical errors or external interference, ensuring that the target is always in the center of the field of view;

[0052] The combination of S14, USB camera, and improved YOLOv8 algorithm demonstrates remarkable performance: in complex backgrounds (such as multi-target occlusion and lighting changes), the improved model's detection accuracy (mAP@0.5) for large targets is 12.8% higher than that of native YOLOv8, and the processing time of a single frame of image meets real-time requirements, ensuring the real-time output of target coordinates and precise coordination of gimbal movement in dynamic scenes, effectively improving the reliability and accuracy of target tracking in complex environments.

[0053] To achieve closed-loop control, the system installs an encoder on the stepper motor. The encoder provides real-time feedback on the motor's speed and displacement, compares this data with the target position, calculates the error, and adjusts the motor's control signal based on the error, thereby achieving precise control of the pan-tilt motion. Closed-loop control can effectively improve the system's tracking accuracy and stability, and reduce tracking deviations caused by external interference or system errors.

[0054] The two-degree-of-freedom pan-tilt structure provides a stable, flexible and precise motion platform for the target tracking system through reasonable mechanical design, precise power drive, effective transmission method, reliable visual acquisition and real-time feedback control, which can meet the needs of target tracking in complex environments.

[0055] The stepper motor has precise position control capabilities and can accurately rotate a certain angle according to the input pulse signal, thereby achieving precise movement of the pan-tilt head. By controlling the rotation of the stepper motor, the pitch and horizontal angles of the pan-tilt head can be precisely adjusted, allowing the visual equipment to accurately align with the target.

[0056] like Figure 3 As shown, the motor control algorithm includes:

[0057] The driver consists of multiple modules marked with "ARDUINO" and modules for setting speed values ​​(such as "50"). The motor's forward and reverse rotation and speed are controlled by setting the pins of the "ARDUINO" module (such as Pin: 3, Pin: 8, etc.);

[0058] The speed acquisition part is mainly composed of the "soEncoder" encoder module and the "double" module. The "soEncoder" module collects the pulse signal sent by the encoder when the motor rotates, and the "double" module processes it.

[0059] During the entire control process, the driver sends control signals to the motor via the "ARDUINO" module based on the set speed value and control logic. The motor's operation drives the encoder to generate pulse signals, which are collected by the "soEncoder" module and processed by the "double" module.

[0060] The processed data is used for feedback control, comparing the actual speed with the set speed. If there is a deviation, the control signal is automatically adjusted to achieve precise control. In actual testing, the Simulink in-the-loop simulation function can be used to directly modify the set speed value (for example, from 50 to 60) to observe the actual motor speed changes, thereby verifying the effectiveness of the control algorithm and in-the-loop simulation function.

[0061] like Figure 4-7 The algorithm shown is based on sinusoidal acceleration and deceleration and integrates PID, sliding mode, and fuzzy control and is burned to the STM32 development board. It is necessary to first establish a mathematical model of the stepper motor. To establish a mathematical model of the stepper motor, the transfer function is:

[0062]

[0063] In the above formula, u represents voltage, i represents current, and R represents resistance;

[0064] Establish its mathematical model and express it in the form of differential equation:

[0065]

[0066] For the 573S09 stepper motor, derive its transfer function:

[0067]

[0068] The model incorporates the moment of inertia J, viscous damping coefficient KD, phase inductance L1, phase current i a 、Number of teeth Z rThe moment of inertia reflects the inertia of the motor rotor and load, the viscous damping coefficient reflects the friction resistance characteristics, and the constant term in the numerator and denominator comprehensively reflects the coupling relationship between electrical and mechanical parameters, which jointly determine the second-order dynamic characteristics of the system (such as natural frequency and damping ratio), providing a key dynamic model foundation for subsequent controller design.

[0069] 1. Substitute the motor parameters and get With the standard formula (Well-known) comparison yields a=1, b=0.233, c=86.771, and these parameters are substituted into the state space model Get specific

[0070] 2. The matrix elements in the state space model are directly derived from the coefficients of the transfer function, such as -a / c==-86.711, -a / b=-0.233, c / a=86.771. These parameters are used to calculate the control law u in the subsequent sliding mode controller design. For example, in the sliding mode controller expression N1=86.771, N2=0.233 are directly taken from the numerator and denominator coefficients of the transfer function.

[0071] 3. In Simulink simulation, the stepper motor transfer function G(s) acts as the controlled object and exists in the control system model as a transfer function module. Connecting to the controller: This module is connected to the output of a sliding mode controller or fuzzy PID controller. The controller's output signal u(t) serves as the stepper motor input. The motor's output (e.g., angle θ) is fed back into the control system, forming a closed-loop control loop.

[0072] 4. The denominator polynomial s of the transfer function 2 +0.233s+86.771 determines the characteristic roots of the system, which are a pair of conjugate complex roots with negative real parts, indicating that the system is stable and the response has oscillatory characteristics.

[0073] Dynamic performance indicators: The parameters of the transfer function directly affect the dynamic performance of the system, such as the damping ratio ζ≈0.026, the natural frequency ω n ≈9.31, which determines the system's performance indicators such as rise time and overshoot.

[0074] Then, in Simulink, the sinusoidal acceleration and deceleration control module, PID control module, sliding mode control module and fuzzy control module are constructed respectively;

[0075] The sliding mode controller is designed based on the sliding mode control theory, by switching the function and exponential reaching law Realize robust control, suppress system chattering, and combine Lyapunov function To verify the stability, the sliding mode controller expression is specifically:

[0076]

[0077] Among them, N1=86.771, N2=0.233, this expression is derived by establishing the state space model of the stepper motor and combining it with the sliding mode control theory. In the design process, the state variable x1=θ(t) is first defined. , constructing a state space model y=[1 0]x+[0]u(t) Then select the switching function According to the exponential reaching law This controller expression is designed to realize sliding mode control of the stepper motor and ensure the robustness and dynamic performance of the system. In Simulink simulation, in order to facilitate parameter setting, the controller output is designed to be u=f cn (d_r,dd_r,r,x1,x2,c,delta), its internal program realizes the function of sliding mode controller through specific parameter calculation and logic.

[0078] Design a fuzzy PID controller, taking the error e and the error change rate ec as input, and dynamically adjust the proportional gain K P , integral gain K i and differential gain K d , dynamic adjustment is made through the fuzzy rule table, the membership function adopts S-type, generalized bell-shaped and triangle functions, and the fuzzification is performed through the membership function and the centroid method.

[0079] In essence, the traditional PID controller (u(t)—control quantity; e(t)—control error at the current moment, that is, e(t)=r(t)-y(t); K p , K i , K d ——corresponding to the coefficients of proportion, integration and differentiation respectively. ) Based on this, fuzzy logic is used to calculate the parameter K p , K i , K d In practical applications, by fuzzifying the system error e and the error change rate ec, as shown in Table 1, according to the pre-set fuzzy rule table, K p , K i , K d When determining the fuzzy language variables, the deviation value e and the change ec are input, the motor pulse control value is output, and the output is determined by the proportional gain K p , integral gain K i , differential gain Kd By selecting appropriate membership functions (such as S-type, generalized bell-shaped, Z-type, and triangular membership functions) to fuzzify the input and output variables, and then using fuzzy reasoning and defuzzification methods to obtain accurate parameters, the controller output can be optimized in real time, so that the system can maintain good control performance under different working conditions.

[0080] Dynamic adjustment mechanism of sliding mode controller parameters: The sliding mode controller derives parameter constraints through Lyapunov stability theory, sets the switching gain σ∈[10,50] and the reaching law coefficient q∈[0.1,2], and dynamically optimizes the parameters through the adaptive adjustment formula σ(k+1)=σ(k)+η·|s(k)| (η is the learning rate, ranging from 0.01 to 0.1) to ensure And δ≥0.5, suppressing the vibration amplitude to less than 0.1rad / s;

[0081] Multidimensional rule design of fuzzy PID controller: The fuzzy rule table is designed based on the quantization levels of error e and error change rate ec (NB = -3, NM = -2, NS = -1, Z = 0, PS = 1, PM = 2, PB = 3), and contains 49 control rules.

[0082] Table 2-2 Fuzzy rule table

[0083]

[0084] Table 1

[0085] The sinusoidal acceleration and deceleration algorithm is used as the motor operation program, through the sinusoidal function:

[0086]

[0087] v(t): represents the instantaneous speed of the stepper motor at time t. It is a function of time t and reflects the change in motor speed over time during motion. In sinusoidal acceleration and deceleration control, the motor speed changes according to the law of the sine function, gradually increasing with time t during the acceleration phase, reaching a maximum value, and then gradually decreasing during the deceleration phase.

[0088] v max Vmax represents the maximum speed of a stepper motor during operation, representing the upper limit of the motor's achievable speed. This parameter is crucial for determining the motor's performance and efficiency. It determines the motor's speed during uniform motion and also influences the operating speed and duty cycle of the entire system. In practical applications, the selection of Vmax requires comprehensive consideration of factors such as motor performance, load conditions, and system accuracy requirements.

[0089] t: A time variable used to describe a specific moment in the motor's motion. In sinusoidal acceleration and deceleration control, time t begins when the motor begins accelerating or decelerating. The change in t determines the motor's speed, V(t), at different times.

[0090] T is the duration of the acceleration or deceleration process. During the acceleration phase, t gradually increases from 0 to T, and the motor speed V(t) gradually increases from 0 to Vmax according to the law of the sine function. During the deceleration phase, t also starts from 0 (re-timing), and the motor speed v(t) gradually decreases from Vmax to 0 according to the law of the sine function. The length of T directly affects the smoothness of the motor acceleration and deceleration process and the response speed of the system. A shorter T will make the motor accelerate and decelerate faster, but may cause the motor to move unsteadily and produce greater shock. A longer T will make the motor acceleration and deceleration process smoother, but will prolong the entire motion cycle.

[0091] Dynamically adjust the pulse frequency to reduce mechanical vibration and loss of step, and improve motion smoothness; integrate sliding mode control, PID control, and fuzzy control algorithms to form a composite control strategy: use PID control in the system stable range to improve accuracy; switch to sliding mode control in nonlinear or strong interference ranges to enhance robustness; use sliding mode control in the event of large disturbances or parameter changes. Then build a complete simulation model, set simulation parameters and run the simulation. Simulate different working conditions to evaluate the algorithm performance, and adjust parameters for optimization based on the results. Finally, use Simulink's automatic code generation function to convert the optimized algorithm into embedded C language code. Configure the development board environment by installing STM32CubeMX software, and burn the code to the STM32 development board to achieve effective control of the stepper motor.

[0092] The sliding mode controller is derived by setting the Lyapunov function It proves that the system is stable. Its design based on sliding mode surface makes the system unaffected by model parameter changes and external interference under nonlinear or strong interference, and can tend to stable sliding mode. The fuzzy PID controller uses fuzzy logic to calculate the PID parameter K. P , K I , K D Online adjustment uses fuzzy rule tables to dynamically correct parameters based on different errors and error change rates, enhancing system stability and adaptability, and can effectively cope with nonlinearity and strong interference.

[0093] Experimentally, the stability of the system under different control algorithms was verified by collecting and analyzing the motor speed and displacement curves under different acceleration and deceleration methods, such as sinusoidal acceleration and deceleration experiments. The experimental results show that after adding the control algorithm, the motor motion accuracy is improved, and it can effectively deal with nonlinear and strong interference factors such as the motor's own accuracy limitations, inaccurate device assembly, and experimental environment interference, ensuring stable operation of the system.

[0094] like Figure 8 As shown, the improved target tracking system algorithm uses the YOLOv8 algorithm and performs processing:

[0095] First, YOLOv8 is selected as the basic algorithm for target recognition because of its fast detection speed and high accuracy, which makes it very suitable for real-time target tracking systems.

[0096] Next, we collected and annotated a dataset. Using water cups as the target, we used a mobile phone or camera to capture images of water cups at different angles and lighting conditions to ensure the diversity of the dataset and improve the model's generalization ability.

[0097] Then use labeling tools such as LabelImg to annotate the image. The annotation content includes the bounding box location and category information of the target object and is saved in YOLO format. The annotated dataset is then divided into training set, validation set and test set.

[0098] During the model training phase, first install the PyTorch framework and its related dependent libraries, and prepare hardware resources such as a computer equipped with an RTX4060Ti graphics card and a 13500hx CPU;

[0099] Configure the YOLOv8 model parameters based on the characteristics of the dataset and task requirements. For example, set cache to false, imgsz to 640, close_mosaic to 0, workers to 0, and batchsize to 1. Train for 200 epochs, then retrain the model on the training set and evaluate its performance on the validation set, observing changes in metrics such as accuracy, recall, and mAP.

[0100] The YOLOv8 model training method includes:

[0101] The data enhancement and model training process uses the Labelimg annotation tool to construct a dataset containing 5,000 images, annotated as "target objects", and uses preprocessing strategies such as Mosaic data enhancement, random flipping, and Gaussian blur. The dataset is divided into a training set of 3,500 images, a validation set of 1,000 images, and a test set of 500 images in a ratio of 7:2:1. The PyTorch framework is used for training for 200 epochs, the optimizer is AdamW, the initial learning rate lr = 0.001, and the learning rate decay strategy is cosine annealing. The formula is:

[0102]

[0103] where lr min =1e-5, T max =200;

[0104] To further improve the performance of the model, the P6 layer is introduced into the backbone network to enhance the detection capability of large targets. The C2f module and SPPF module are combined to enhance the feature extraction capability. The P6 convolution downsampling formula is:

[0105] F P6 =Conv 3×3，s＝2 (F P5 )

[0106] The C2f module formula divides the input feature map into two parts: A and B, where A is processed by deep convolution and B is directly spliced.

[0107] F out =Concat(Conv(A),B)

[0108] Improved detection heads. In addition to the existing P3, P4, and P5 detection heads, a new P6 detection head is added to process 1 / 64 downsampled feature maps. ChannelTransformer and ConvFormer technologies are used to optimize multi-scale feature fusion. By modeling and interacting with feature channels, feature information at different scales is fully integrated to improve detection accuracy.

[0109] Enhance the correlation between low-level and high-level features by adaptively performing cross-channel feature interaction in the channel dimension; input feature maps F1, F2, ..., Fn of different scales;

[0110] Use the following formula:

[0111]

[0112] In the above formula, Q, K, and V are query, key, and value, respectively, which are channel features from different scales. Through the cross-attention mechanism, the information between channels can be fused;

[0113] The fused multi-scale features are more finely aligned with the features of the decoder stage to solve the consistency problem of semantics at different levels;

[0114] Fusion of features from different stages by computing the relationship between query Q and key K;

[0115] Use the following formula:

[0116]

[0117] Among them, Q comes from the fusion features of the encoder, and K comes from the features of the decoder.

[0118] like Figure 9As shown, it was then optimized through ablation experiments. Experiments were designed to compare the performance of the original YOLOv8n model, the YOLOv8n-P6 model after introducing the P6 layer, the YOLOv8-P6-Ctrans model combined with the C2f and SPPF modules, and the YOLOv8-P6-Ctrans-Con vFormer model using ChannelTransformer and ConvFormer technology;

[0119] Use the test set to evaluate each model and analyze the impact of each improvement on performance to determine the optimal model structure;

[0120] Finally, the improved and optimized YOLOv8 model was combined with the DeepSORT algorithm. YOLOv8 was responsible for detecting the location and category of the target object, while the DeepSORT algorithm tracked the target, assigned a unique ID, and predicted the motion trajectory based on the detection results. The fused algorithm was then deployed to the USB camera system.

[0121] The camera captures images in real time, and displays the results after YOLOv8 detection and DeepSORT tracking. At the same time, the target coordinate information is sent to the STM32 development board through the serial port to control the two-degree-of-freedom gimbal movement to keep the target in the center of the camera's field of view.

[0122] Ablation experiment and performance comparison,The effectiveness of each improved module is verified through ablation experiments, and the results are shown in the following table:

[0123]

[0124] After introducing the P6 layer, ChannelTransformer, and ConvFormer, mAP@0.5 increased by 12.8%, mAP@0.95 increased by 6.6%, and the number of parameters only increased by 44.1%, verifying the effectiveness and lightweight advantages of the improved module.

[0125] It should be noted that, in this document, terms such as "myself" and "cup" are used only to distinguish this document from others, and do not necessarily require or imply any actual relationship between these entities or operations; moreover, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus;

[0126] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0127] The present invention and its embodiments are described above. Such description is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in this field are inspired by it and do not depart from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical solution, which should fall within the scope of protection of the present invention.

Claims

1. A target tracking method integrating visual perception and motion control, characterized in that: Including steps: S1. Improve the YOLOv8 model and pre-train the improved YOLOv8 model; The improved YOLOv8 model specifically introduces the P6 layer in the backbone network, extracts global features of large objects through 3×3 convolution downsampling and the C2f module; integrates the ChannelTransformer module in the neck network, and solves the problem of multi-scale feature semantic gaps through a cross-channel attention mechanism; and integrates the ConvFormer module in the detection head, combining the Transformer self-attention mechanism to enhance the ability to capture long-range dependencies. S2, after preprocessing, the image captured by the camera is input into the YOLOv8 model trained in step S1, and the target bounding box and the pixel coordinates of the target center (x i ,y i ); S3, the target bounding box and target center pixel coordinates (x i ,y i ) is transferred to the development board of the pre-burned algorithm program, where x i ,y i Respectively represent the horizontal and vertical coordinates of the target center in the image coordinate system; the development board is designed based on the mathematical model of the stepper motor and is based on sinusoidal acceleration and deceleration. It integrates PID control, sliding mode control and fuzzy control algorithms, and generates code for burning after in-loop simulation optimization; S4, the development board calculates the target boundary box and the target center pixel coordinates (x i ,y i ) generates control instructions to drive the stepper motor to adjust the gimbal posture so that the target remains in the center of the field of view and then stops moving.

2. The target tracking method integrating visual perception and motion control according to claim 1, characterized in that: The step S1 pre-trains the improved YOLOv8 model as follows: A dataset of 5,000 images was constructed using the Labelimg annotation tool, with the labeling category "target object." Mosaic data augmentation, random flipping, and Gaussian blurring were used as preprocessing strategies. The dataset was divided into training, validation, and test sets in a ratio of 7:2:

1. The PyTorch framework is used for training for 200 epochs. The optimizer is AdamW, the initial learning rate lr = 0.001, and the learning rate decay strategy is cosine annealing. The formula is: Where, lr min =1e-5, T max =200.

3. The target tracking method integrating visual perception and motion control according to claim 1, characterized in that: The specific steps of pre-burning the algorithm program into the development board in step S3 are as follows: S3.1, establish the mathematical model of the stepper motor, the transfer function is: In the formula, u represents voltage, i represents current, and R represents resistance; Establish a mathematical model of the stepper motor and express it in the form of a differential equation: The transfer function of the stepper motor is derived as: Among them, Z r is the number of teeth of the stepper motor, L1 is the phase inductance, i a is the phase current, J is the moment of inertia, K D is the viscous damping coefficient; S3.2, according to the stepper motor transfer function obtained in step S3.1, design a sliding mode controller based on sliding mode control theory, and switch the function and exponential reaching law Realize robust control, suppress system chattering, and combine Lyapunov function To verify the stability, the sliding mode controller expression is specifically: Where N1 and N2 are taken from the numerator and denominator coefficients of the stepper motor transfer function respectively; S3.3, design a fuzzy PID controller with error e and error change rate ec as input, motor pulse control value as output, and dynamically adjust proportional gain k P , integral gain k i and differential gain K d , by selecting membership functions to fuzzify the input and output variables, and then obtaining accurate parameters based on fuzzy reasoning and defuzzification methods, thus achieving real-time optimization of the controller output; S3.4 uses a sinusoidal acceleration and deceleration algorithm as the motor operation program. The motor pulse control value is dynamically adjusted through a sinusoidal function. A fuzzy PID controller is used in the system stability range, and a sliding mode controller is used in the nonlinear or strong interference range. The sinusoidal acceleration and deceleration algorithm is specifically as follows: Where v(t) represents the instantaneous speed of the stepper motor at time t, v max Indicates the maximum speed of the stepper motor during operation.

4. The target tracking method integrating visual perception and motion control according to claim 1, characterized in that: The specific process of step S4 is as follows: The center of the camera imaging plane is taken as the origin (x c ,y c ), establish a two-dimensional coordinate system, the target center pixel coordinate (x i ,y i ) and the origin (x c ,y c ) of the deviation Δx=x i -x c , Δy=y i -y c Reflects the position offset of the target in the field of view; if Δx>0, the gimbal rotates horizontally to the left; if Δx<0, it rotates horizontally to the right; if Δy>0, the gimbal rotates downward in pitch; if Δy<0, it rotates upward in pitch; The dead zone range is preset [-T, T]. When the deviation values ​​Δx and Δy both fall into the preset dead zone range, the motor stops moving to achieve stable target tracking.

5. The target tracking method integrating visual perception and motion control according to claim 1, characterized in that: In step S2, the DeepSORT algorithm is used to associate the targets in the previous and next frames based on the continuous frame images captured by the camera, the target trajectory is predicted by Kalman filtering, and the appearance features and motion features are matched by the Hungarian algorithm.

6. The target tracking method integrating visual perception and motion control according to claim 3, characterized in that: The sliding mode controller derives parameter constraints through Lyapunov stability theory, sets the switching gain σ∈[10,50] and the reaching law coefficient q∈[0.1,2], and dynamically optimizes the parameters through the adaptive adjustment formula σ(k+1)=σ(k)+η·|s(k)|, where η is the learning rate, which takes a value of 0.01-0.1, to ensure And δ≥0.5, suppressing the vibration amplitude to less than 0.1rad / s; The fuzzy PID controller is designed based on the quantization levels NB = -3, NM = -2, NS = -1, Z = 0, PS = 1, PM = 2, and PB = 3 of the error e and the error change rate ec, and contains 49 control rules.

7. The target tracking method integrating visual perception and motion control according to claim 5, characterized in that: The Hungarian algorithm is used to match the cost matrix C, combined with the Mahalanobis distance d Mahalanobis and cosine distance d cosine , the formula is: C = 0.9d Mahalanobis +0.1d cosine , where the Mahalanobis distance measures the consistency of the target motion features, and the cosine distance measures the similarity of the target appearance features.

Images