Target identification tracking method based on dynamic scene perception of inspection robot

Abstract

The invention discloses a target identification tracking method based on dynamic scene perception of an inspection robot, and the method specifically comprises the following steps: 1, building an automatic hog house inspection system, collecting inspection video data, and obtaining real hog house video data; step 2, carrying out preprocessing on the pigs-ywuzm public data set; step 3, preprocessing the real pig house video data; 4, constructing a multi-source integrated data set, and performing enhancement processing on the multi-source integrated data set; step 5, constructing an identification tracking model; step 6, processing the real pig house video data obtained in the step 1; and step 7, training the identification tracking model constructed in the step 5 by using the data set enhanced in the step 4, and testing the trained model by using the data set processed in the step 6. According to the invention, the problems of low target detection accuracy and poor tracking continuity caused by limited image acquisition are solved.

Description

Technical Field

[0001] This invention belongs to the technical field of perception, computer vision and robot system integration, and relates to a target recognition and tracking method based on dynamic scene perception of inspection robots. Background Technology

[0002] In the development of modern animal husbandry, traditional manual patrols or fixed-point data collection methods for identifying and monitoring livestock behavior on pastures are inefficient and subjective, making them unsuitable for large-scale, 24 / 7 management needs. Therefore, current methods largely rely on fixed cameras for image acquisition. However, this approach suffers from limitations such as restricted image viewing angles, insufficient area coverage due to limited camera movement, and poor timeliness. Especially in scenarios with frequent livestock activity, large-scale motion blur, complex occlusion, or image distortion interference, traditional image recognition often faces decreased accuracy and interrupted target tracking, making it difficult to guarantee target detection accuracy and tracking continuity. Therefore, there is an urgent need to establish an automated data acquisition and target recognition and tracking system for dynamic inspection scenarios to improve perception capabilities and application effectiveness in real pasture environments. Summary of the Invention

[0003] The purpose of this invention is to propose a target recognition and tracking method based on dynamic scene perception of inspection robots, which solves the problems of low target detection accuracy and poor tracking continuity caused by limited image acquisition.

[0004] The technical solution adopted in this invention is a target recognition and tracking method based on dynamic scene perception of inspection robots, which specifically includes the following steps: Step 1: Build an automatic pigsty inspection system and collect inspection video data to obtain real pigsty video data; Step 2: Preprocess the pigs-ywuzm public dataset; Step 3: Preprocess the real pigsty video data; Step 4: Construct a multi-source ensemble dataset and perform augmentation processing on the multi-source ensemble dataset; Step 5: Build the identification and tracking model; Step 6: Process the real pigsty video data obtained in Step 1; Step 7: Use the enhanced dataset from Step 4 to train the recognition and tracking model constructed in Step 5, and use the dataset processed in Step 6 to test the trained model.

[0005] The invention is further characterized by: The specific process of step 1 is as follows: Step 1.1, a closed loop track is erected above the ceiling area of ​​the adjacent pens in the pig house, covering multiple pig pens and activity areas arranged sequentially from pen 1 to pen m. Step 1.2: Integrate an image data acquisition terminal onto the inspection robot body; Step 1.3: Fix the inspection robot with integrated image data acquisition terminal on the circular track built in Step 1.1, and set the default cruising route to a clockwise direction along the track. Step 1.4: During the robot inspection process, the visual acquisition module in the image data acquisition terminal continuously collects videos of the pig herd's behavior throughout the day, receives and stores the inspection video data in real time, and marks the data according to the location of the pigsty to form real pigsty video data covering different time periods and locations.

[0006] The specific process of step 2 is as follows: Step 2.1, obtain the pigs-ywuzm public dataset on the Roboflow platform; Step 2.2: Adjust the image sizes in the public dataset to obtain a standard-sized image dataset; Step 2.3: Perform independent normalization on the color channels of each image in the standard-size image dataset processed in Step 2.2 to obtain the preprocessed pigs-ywuzm public dataset.

[0007] The specific process of step 3 is as follows: Step 3.1, obtain the real pigsty video data obtained in step 1.4; Step 3.2: Use interpolation to count the number of frames in the target image data from the real pigsty video data obtained in Step 3.1. Perform calculations, according to Image segmentation was performed on real pigsty video data to extract valid frame image data. The interpolation calculation formula is as follows: (1) Step 3.3, divide the dataset from Step 3.2 into portions according to an 8:1:1 ratio. The dataset was divided into training, validation, and test sets. The image feature labeling tool LabelImg was used to label the training and validation sets to obtain a real pigsty dataset.

[0008] The specific process of step 4 is as follows: Step 4.1, denoise the pigs-ywuzm public dataset processed in step 2.3 and the real pigsty dataset processed in step 3.3 respectively; Step 4.2: Integrate the public dataset of pigs-ywuzm processed in step 4.1 with the real pig house dataset to obtain a multi-source integrated dataset, and then perform data cleaning on the multi-source integrated dataset; Step 4.3: The multi-source ensemble dataset obtained in Step 4.2 is processed using the Mosaic data augmentation method; Step 4.4: Apply a multi-task learning method based on behavior and scene features to set learning tasks for the multi-source ensemble dataset processed in Step 4.3.

[0009] The specific process of step 4.4 is as follows: Step 4.4.1, conduct a dynamic object action learning task on the pigs-ywuzm public dataset after processing in step 4.3; Step 4.4.2: Perform a scene learning task for dynamic objects on the real pigsty dataset processed in Step 4.3; Step 4.4.3 involves adaptively assigning weights to the learning tasks from steps 4.4.1 and 4.4.2, and then performing a comprehensive modeling of the two types of tasks to obtain the comprehensive task value. The calculation formula is as follows: (2) in, and This represents the task value and weight coefficient in action, behavior, and state learning tasks.

[0010] The specific process of step 5 is as follows: Step 5.1, initialize all modules of the YOLOV8 target recognition model; Step 5.2, adjust the framework structure in the YOLOV8 model, including replacing the detection head structure and introducing a temporal modeling module; Step 5.3: Based on step 5.2, the Leaky ReLU activation function is used instead of the default SiLU activation function in the convolutional layers of the YoloV8 model, while the Sigmoid activation function is retained in the output layer. Step 5.4: The improved model from Step 5.3 is enhanced by combining the Motion Blur and Dynamic Soft Labeling strategies. Step 5.5: Initialize the DeepSort tracking module; Step 5.6: Define and instantiate the DeepSort tracking module class object in the YOLOV8 recognition model after the processing in step 5.4. After each frame of image is recognized by YOLOV8, the detected bounding box and class confidence are passed as input to the DeepSort module to obtain the recognition and tracking model.

[0011] The specific process of step 5.2 is as follows: Step 5.2.1, in the Head part of the YOLOV8 model, the original detection head module of YOLOV8 is replaced with the PP-YOLOEHead lightweight detection structure; Step 5.2.2: A lightweight temporal modeling module (TSM) is introduced at the backbone end of the YOLOV8 model to enhance the 2D convolutional network by processing the feature maps of the frame data through Temporal Shift.

[0012] The beneficial effects of this invention are as follows: By optimizing the target detection and tracking algorithm, it significantly improves the accuracy and continuity of animal identification in complex pastoral environments, solving the problems of low accuracy and easy tracking interruption in traditional image recognition scenarios with multiple occlusions, motion blur, and limited viewpoints. By introducing a lightweight structure and a temporal modeling module, the accuracy and continuity of the model's identification of dynamic targets are improved; combined with multi-task learning and fuzzy adaptation strategies, the robustness of the model in complex scenarios is enhanced. This invention constructs an automated perception system based on a track-mounted inspection robot, realizing multi-angle, cross-regional dynamic image acquisition, improving the spatial coverage and temporal continuity of the overall system. Furthermore, this invention integrates a complete data processing workflow from image acquisition, preprocessing, keyframe selection to sample construction, effectively enhancing the information expression capability of the input image and improving the model's generalization performance and practical deployment capability. Attached Figure Description

[0013] Figure 1 This is the overall framework diagram of the target recognition and tracking method based on dynamic scene perception of inspection robots according to the present invention; Figure 2 This is a schematic diagram of a special ranch scene in which real data is collected in the target recognition and tracking method based on dynamic scene perception of inspection robots in this invention; Figure 3 This is the overall structure diagram of the improved YOLOV8 model in the target recognition and tracking method based on dynamic scene perception of inspection robots in this invention; Figure 4 This is a structural diagram of the TemporalShift module in the target recognition and tracking method based on dynamic scene perception of inspection robots in this invention; Figure 5 This is a framework diagram of the image quality optimization method with multi-step feature enhancement in the target recognition and tracking method based on dynamic scene perception of inspection robots in this invention; Figure 6 This is the result of the YOLOV8 and Deepsort fusion optimization model applied to the optimal image of the pastoral area. Detailed Implementation

[0014] The following detailed description is provided in conjunction with specific implementation methods.

[0015] Example 1 This invention relates to a target recognition and tracking method based on dynamic scene perception of inspection robots, such as... Figure 1The diagram shown illustrates the overall framework of this method, which includes the following steps: Step 1: Build a smart pigsty automatic inspection system and collect inspection video data to obtain real pigsty video data, providing raw data support for subsequent steps. Step 2: Preprocess the pigs-ywuzm public dataset to provide single-source data for the subsequent step 4; Step 3: Preprocess the real pigsty video data to provide single-source data for the subsequent step 4; Step 4: Integrate the single-source data processed in Step 2 and Step 3 to obtain a multi-source integrated dataset. After enhancing the processing of the multi-source integrated dataset, input it into Step 7. Step 5: Optimize the YOLOV8 recognition model and integrate it with the DeepSort tracking module to obtain a recognition and tracking model, providing a model for training and testing in the subsequent step 7; Step 6: Use an adaptive image selection algorithm based on image features and scene to provide image data for training and testing in the subsequent step 7. Step 7: Use the model and data provided in Steps 5 and 6 to train and test the model.

[0016] Example 2 The specific process of step 1 is as follows: Step 1.1, a closed-loop track system is erected along the ceiling area at a height of approximately 2.5 meters above the adjacent pens in the pigsty, such as... Figure 2 As shown, the system covers multiple pig pens and activity areas arranged sequentially from pen 1 to pen m, thereby supporting the inspection robot to achieve continuous area inspection and multi-angle data collection. Step 1.2: An integrated intelligent terminal with edge intelligence processing capabilities is built on the inspection robot body to realize terminal-level data acquisition, perception analysis, and wireless communication functions. This terminal mainly consists of three parts: a wireless communication module, an edge computing module, and a vision acquisition module. The wireless communication module uses an industrial-grade Wi-Fi network card supporting the 5GHz band to achieve stable data interaction with a remote server; the edge computing module uses a Jetson OrinNano embedded AI chip, which has image recognition and real-time inference capabilities; the vision module is equipped with a 360° panoramic fisheye camera to achieve wide-angle coverage and continuous video acquisition of the aquaculture area. First, an embedded motherboard is selected as the core control unit for each module of the terminal, and a lightweight Linux operating system is installed on the motherboard. Second, a CAN bus is used to connect each module to build a basic communication architecture, and a unified power supply is provided through a power management module. Finally, the vision module transmits image data to the edge chip in real time through the MIPI interface for local analysis and processing. The processing results are then transmitted to the motherboard at high speed via the PCIe channel. After the data is transmitted back to the motherboard, the communication module connected via the USB interface completes the stable uploading of high-frequency data. Step 1.3: Fix the inspection robot on the track system, set the default patrol route to a clockwise direction along the track, automatically or manually control the patrol in different areas of the pig house, and set the IP address and port of the remote server on the inspection robot to achieve stable communication with the backend data processing console, supporting the uploading and interaction of real-time monitoring video data; Step 1.4: During the robot inspection, the behavior video of the pig herd is continuously collected 24 hours a day by fisheye camera. The backend console receives and stores the inspection video data in real time, and marks the data according to the location of the pigsty to form real pigsty video data covering different time periods and different locations, which is used for subsequent data preprocessing, model training and target detection tasks.

[0017] Example 3 The specific process of step 2 is as follows: Step 2.1, obtain the pigs-ywuzm public dataset on the Roboflow platform. This dataset has been partitioned and labeled to provide basic data resources for subsequent steps; Step 2.2, uniformly adjust the image size in the public dataset to... This yields a standard-sized image dataset, ensuring data consistency and processing efficiency. Step 2.3: Perform independent standardization on the color channels of each image in the standard-size image dataset processed in Step 2.2 to eliminate the bias between different channels.

[0018] Example 4 The specific process of step 3 is as follows: Step 3.1, obtain the real pigsty video data obtained in step 1.4 to provide basic data resources for subsequent steps, so as to enhance the model's ability to judge the actual environment; Step 3.2: Calculate the number of frames in the target image data using interpolation. ,according to Image segmentation was performed on real pigsty video data to extract valid frame image data. This is to facilitate the analysis and processing in step 3.3. The interpolation calculation formula is as follows: (1) in, , , To the number of images to be cut, These are real pigsty video data from various time periods, and Corresponding to the start time and start frame in the video. Corresponding to the end time and end frame in the video; Step 3.3, divide the dataset from Step 3.2 into portions according to an 8:1:1 ratio. The dataset is divided into training, validation, and test sets. The image feature labeling tool LabelImg is used to label the training and validation data to obtain a real pigsty dataset. The labeling information is saved as a txt file conforming to YOLO format to ensure data format consistency.

[0019] Example 5 The specific process of step 4 is as follows: Step 4.1, apply an adaptive improved Gaussian filtering algorithm to denoise the pigs-ywuzm public dataset processed in step 2.3 and the real pigsty dataset under the fisheye camera processed in step 3.3, respectively, in order to reduce the interference introduced by noise during training and improve image quality; Step 4.1.1, calculate the variance in a certain region of the image. The calculation formula is as follows: (2) (3) in, Indicates the center point Nearby Scope of the field, and express pixel information, This represents the average value of the information from each pixel within the pixel matrix. Variance The larger the value, the greater the dispersion of the pixel matrix in that region, and the more it needs to be reduced. Conversely, it is necessary to increase the size. , The standard deviation of the Gaussian kernel; Step 4.1.2, calculate the variance obtained in step 4.1.1. With two-dimensional Gaussian filter function By comparing, the function is obtained. As shown in formula (4) below, Represents the coordinates of the center pixel within the convolution window: (4) Step 4.1.3, will Treat it as a constant. This is a concept related to the Gaussian kernel radius. with standard deviation The function is used to obtain the final result, and the calculation formula is: (5) Among them, when At that time, pixel matrix regional variance That is, the degree of dispersion of gray values ​​in this region and the two-dimensional filtering function. The degree of dispersion is equal at each point, meaning the weights of the parameters in the Gaussian kernel are equal to... The pixel matrix grayscale value weights of the regions are closest. Since a Gaussian kernel size that is too large has very small edge weights during the filtering process, it is practically meaningless. Therefore, a smaller Gaussian kernel size is usually chosen. or Gaussian kernel size.

[0020] Step 4.2 integrates the pigs-ywuzm public dataset and the real pig farm dataset processed in Step 4.1, and then performs data cleaning on the multi-source integrated dataset, including removing duplicate images, eliminating low-quality or invalid images, and filling in missing annotation information to improve the accuracy and effectiveness of the data.

[0021] Step 4.3: The multi-source ensemble dataset obtained in Step 4.2 is processed using the Mosaic data augmentation method to improve the generalization ability and robustness of the model. Step 4.3.1: Randomly select four original images from the training and validation sets of the multi-source ensemble dataset, and randomly determine the coordinates of a reference point. As a reference position for splicing; Step 4.3.2: Perform a series of data augmentation operations independently on each selected image, including horizontal flipping, resizing, and color gamut transformation. The formula for horizontal flipping can be expressed as: (6) in, These are the rotated coordinates. These are the coordinates before rotation. It is the center of rotation. It is the angle of rotation. This refers to the coordinates of the top-left corner of the rotated image. The formula for scaling can be expressed as: (7) in, These are scaled coordinates. These are the coordinates before scaling. and The scaling factor is used. After the above processing, four augmented images are obtained, significantly improving the diversity and richness of the data; Step 4.3.3: Based on the reference points selected in Step 4.3.1, the four augmented images processed in Step 4.3.2 are combined into a new image of uniform size through random cropping, arrangement, and scaling operations. The layout rules are as follows: the first image is located in the upper left region of the new image, the second image is located in the upper right region, the third image is located in the lower left region, and the fourth image is located in the lower right region. Simultaneously, it is ensured that the object detection box labels and category labels corresponding to the four original images selected in Step 4.3.1 are migrated to the new image along with their respective sub-image regions, maintaining the consistency of the label information. Step 4.3.4: For each target detection box migrated to the new image in Step 4.3.3, based on the coordinate transformation relationship of its sub-image region (i.e., the operation in Step 4.3.2) and combined with the position offset of the stitching layout, the absolute coordinates of the detection box in the new image coordinate system are recalculated according to the following formula: (8) in, This represents the sub-image position offset. A boundary truncation strategy is then applied to handle the bounding boxes that extend beyond the stitched image boundaries; the processing formula is as follows: (9) Restrict the coordinates to a valid range, where and These are the width and height of the stitched image, respectively. Additionally, invalid detection boxes with a truncated width or height less than 5 pixels are deleted to ensure annotation quality and training accuracy. Step 4.3.5: Standardize the new image obtained in step 4.3.4, including pixel value normalization and image size adjustment. The Z-score standardization formula is shown below: (10) The normalization formula is shown below: (11) in, This represents the normalized value. This represents the original data value. This represents the minimum value in the data. This represents the maximum value in the data. Simultaneously, the stitched images are uniformly scaled to [a specific value]. The model input size in pixels. This represents the standardized value. This represents the original data value. This represents the mean. It represents the standard deviation.

[0022] Step 4.4: Use a multi-task learning method based on behavior and scene features to set learning tasks for the multi-source ensemble dataset processed in Step 4.3, so that the model can capture the correlation between multiple tasks at the same time and improve the training effect of the model in Step 7.

[0023] Step 4.4.1: On the pigs-ywuzm public dataset processed in step 4.3, carry out the learning tasks of the actions (e.g., "drink", "eat", "fight", "lie down", "jump", "run", "sit", "sleep", "stand" and "walk"), habits (e.g., "nose to nose", "poke other places with nose" and "play with toys") and states (e.g. "other" states) of dynamic objects. Step 4.4.2: On the real pigsty dataset processed in Step 4.3, a learning task is carried out on the circular living scene of adjacent pigsties with dynamic objects and the camera scene under the fisheye camera. Step 4.4.3 involves adaptively assigning weights to the learning tasks from steps 4.4.1 and 4.4.2, and performing a comprehensive modeling of the two types of tasks to calculate the total task value. This formula reflects the weighting mechanism and the regulatory role of interference factors in the task fusion process, which helps to optimize and adapt learning tasks to complex environments. The specific calculation formula is as follows: (12) in, This represents the overall task value. and This represents the task value and weight coefficient in action, behavior, and state learning tasks. and This represents the task value and weight coefficient in learning tasks within real-life and video scenarios. This represents the moderating term for other potential influencing factors.

[0024] Example 6 The specific process of step 5 is as follows: Step 5.1, initialize all modules of the YOLOV8 target recognition model to ensure the correct configuration and functional implementation of each part of the model; Step 5.2, adjust the framework structure in the YOLOV8 model, including replacing the detection head structure and introducing a temporal modeling module. For example... Figure 3 As shown, based on the basic architecture of YOLOV8, its Backbone, Neck and Head parts have been given more suitable lightweight structures and strategies to optimize performance and improve the accuracy and efficiency of target recognition. Step 5.2.1: In the Head section, the original detection head module of YoloV8 is replaced with the PP-YOLOE Head lightweight detection structure; Step 5.2.1.1: Input the multi-scale fused features P3, P4, and P5 from Backbone and Neck outputs into the spatial channel fusion module. The original feature map size is [size missing]. The features are first extracted into channel description vectors using global average pooling. , describing vector Then, two fully connected layers are used to build non-linear channel dependencies, and channel attention weights are generated using the Sigmoid function. Finally, the generated weight vector is multiplied element-wise with the original feature map along the channel dimension to generate a weighted feature map, which strengthens the expressive ability of local dynamic response regions, thereby improving the model's attention to and discrimination of key target regions. Step 5.2.1.2 involves feeding the weighted feature maps obtained in step 5.2.1.1 into the classification and regression branches for independent modeling. In the classification branch, the weighted feature maps first undergo a... The system consists of convolutional layers, followed by BatchNormalization layers for normalization, Leaky ReLU activation to introduce non-linear feature responses, and finally a... The convolutional layer outputs a class confidence vector for each anchor point. In the regression branch, the weighted feature map extracts positional features through different convolutional kernels, also including BN layers and Leaky ReLU activation for processing, ultimately outputting four coordinate regression parameters: center point offset. , With width and height scaling factors , as well as Fractions are used to depict the geometric shape of an object in space. Step 5.2.1.3 employs a decoupled head architecture, combining the classification branch output and regression branch output obtained in step 5.2.1.2 at the end of the detection head to form a standard target bounding box output format. The output is then fed into the post-processing module, where... A confidence level is established for the target. In the post-processing module, a confidence threshold is applied to remove low-quality target boxes, and a non-maximum suppression algorithm is used within each category, based on the relationships between predicted boxes. The values ​​are subjected to overlap suppression processing, and the retained detection boxes are combined into the final output result.

[0025] Step 5.2.2: Introduce the lightweight timing modeling module TSM (Temporal ShiftModule) at the end of the Backbone, such as... Figure 4 As shown, the 2D convolutional network is enhanced by processing the feature maps of frame data through Temporal Shift, thereby improving the model's ability to model temporal information during object motion.

[0026] Step 5.2.2.1, convert the input 4D feature tensor The channel dimension along The dimension is proportionally divided into three parts. One part of the channel values ​​is shifted forward one frame, another part of the channel values ​​are shifted backward one frame, and the remaining part remains unchanged, forming a new feature tensor. Input is given to subsequent modules; Step 5.2.2.2 introduces a feature interaction mechanism in the time dimension, for each time interval... Feature tensor ,along Dimensions split it into three groups of sub-features In the forward section of the channel In the middle, replace the channel data with time. The eigenvalues ​​of the corresponding channels, in the later channels In the middle, replace the channel data with time. The characteristic values ​​of the corresponding channel, while the static channel part Remain unchanged; Step 5.2.2.3: In the model inference stage, the motion trajectory cues of the target in consecutive frames are modeled to enhance the model's ability to identify fast-moving pigs and its temporal consistency performance, thereby improving the overall robustness of detection.

[0027] Step 5.3: In the convolutional layer, the Leaky ReLU activation function is used instead of the default SiLU activation function, while the Sigmoid activation function is kept in the output layer. By combining the Leaky ReLU activation function in the convolutional layer and the Sigmoid activation function in the output layer, the model's ability to extract temporal and spatial features in dynamic object recognition and tracking tasks is enhanced. Step 5.3.1: First, in the shallow feature extraction stage (Shallow Layers) of the Backbone, the original activation function is replaced with the Leaky ReLU activation function, that is, LeakyReLU (negative_slope=0.1) is applied after each convolutional module. The calculation formula is as follows: (13) in, It is represented as the leakage coefficient, which is a very small positive number, usually set to 0.01; The input value is the linear transformation output of a certain layer in the neural network. This enhances the responsiveness to fine-grained information such as low-level textures and edges, and improves the stability of gradient propagation. Step 5.3.2, next, in the deep feature extraction stage of the backbone (Deeper Layers), the SiLU activation function is continued to be used, that is, a function defined by the following formula is immediately followed by each linear transformation output: (14) in The input value is the linear transformation output of a certain layer in the neural network; Represented as the Sigmoid function, its output ranges from (0,1) and has a non-zero derivative with respect to all real numbers. This is particularly useful in the Neck or Head to maintain non-linear expressiveness and enhance the modeling of high-level semantic features. Step 5.3.3: Then, in the output layer of the detector head, the Sigmoid activation function is used to perform independent binary classification for each category. That is, the existence probability of each category is obtained by applying the following formula to each element of the category vector output by each predicted box. : (15) in, The input value is the linear transformation output of a certain layer in the neural network. It abandons the traditional Softmax mechanism, thus preserving the expressive ability of multi-label co-occurrence between targets, adapting to multi-target co-occurrence scenarios such as overlapping pig groups and occlusion, and avoiding the false exclusion of targets; Step 5.3.4, finally, uniformly set the final output structure as follows: ,in For batch size, For the number of prediction boxes, 5 represents the number of categories. , respectively represent the center coordinates of the target box. Width and height And the confidence level of the target. It adapts to multi-label loss functions to ensure consistency of training objectives during forward inference and backpropagation.

[0028] Step 5.4: Motion Blur and Dynamic Soft Labeling strategies are combined to enhance the model’s performance in real-world scenarios and improve the model’s ability to learn motion continuity. Step 5.4.1: In the data preprocessing stage, a set of directional motion blur kernel templates with horizontal and vertical orientations, diagonal directions, and lengths ranging from 3 to 15 pixels are constructed. A directional blur kernel is randomly applied to each training image with a probability of 30% to 40%. This blur kernel can be defined according to the following formula: (16) in, The length of the fuzzy kernel. It is the direction angle. Represented as pixel values ​​of kernel coordinate offset. It simulates the actual motion blur effect produced by the target or camera device, thereby enhancing the temporal perturbation diversity of the training samples; Step 5.4.2: During model training, estimate the inter-frame displacement velocity of the target in the target tracking or detection labels. This is converted to the fuzzy kernel size in step 5.4.1, and the velocity estimation formula is as follows: (17) in, Indicates the target object in the current frame (time). The coordinates of the center point in the image coordinate system. Indicates the target object in the previous frame (time). The coordinates of the center point in the image coordinate system. This represents the pixel-to-actual distance conversion coefficient, which is obtained through camera calibration. This represents the video capture frame rate. In the training data, a larger blur kernel is used for high-speed target regions, while maintaining the original sharpness for low-speed target regions. This further simulates image blurring caused by camera shake or high-speed target movement in real-world scenarios, thereby improving the model's ability to model images under blurred conditions. Step 5.4.3: During the testing phase, a blurred image enhancement evaluation set is introduced. This set contains image copies with three levels of motion blur: mild, moderate, and severe. Different blurred versions of the same image are input into the model using Test-Time Augmentation (TTA), and the output results are fused. The fusion calculation formula is as follows: (18) in, This represents the model's predicted output for slightly, moderately, and severely blurred images, while The weights can be set to equal weights or learned, representing the fusion weights for different levels of blur. This makes the model more adaptable to complex visual interference such as edge blur and motion blur, thereby improving its robustness in detecting motion-blurred targets. Step 5.4.4: During the label generation process, a confidence adjustment mechanism based on image blurriness is added to the detected target. The blurriness is first calculated using the Laplacian variance index of the image, and the calculation formula is as follows: (19) in, For pixel values, For the Laplace operator, Output the mean value for the operator. The image size is defined as . Images with a calculated value less than 20 are considered highly blurred. For detected targets within highly blurred images, their label confidence is calculated. The value is adjusted from the original 1.0 to a value between 0.7 and 0.9, using the following formula: (20) in, It is a coefficient that controls the descent speed, and This is represented as a high fuzziness threshold. Indicates when (In cases of high ambiguity) this value > 0, leading to a decrease in confidence level. This indicates the range of confidence adjustment values ​​for the soft label. This reduces overfitting to ambiguous targets. Step 5.4.5: Embed the soft label strategy from Step 5.4.4 into the loss calculation module, and introduce BCE Loss (Binary Cross Entropy) as the main loss function for the classification branch, i.e., the soft label confidence. Replace the standard label values ​​(usually 0 or 1) and use the F.binary_cross_entropy() function to perform forward loss propagation. The calculation formula is as follows: (twenty one) in, This is the loss value. This outputs the classification probability values ​​predicted by the model. Simultaneously, the classification loss coefficients in the loss weight configuration file hyp.yaml in the main structure are adjusted to accommodate more lenient label supervision methods, improving training stability and generalization ability.

[0029] Step 5.5: Initialize the DeepSort tracking module to provide the necessary functional support for the object tracking task; Step 5.6: Define and instantiate the DeepSort tracking module class object in the YOLOV8 recognition model. After each frame of image is recognized by YOLOV8, the detected bounding box and class confidence are passed as input to the DeepSort module. The latter further completes the target allocation and trajectory maintenance based on the time series information, thereby realizing the collaborative processing of detection and tracking.

[0030] Example 7 The specific process of step 6 is as follows: Step 6.1, obtain the real pigsty video data obtained in step 1.4 in order to provide the necessary input for subsequent image processing and analysis; Step 6.2: Using an optimal image localization and extraction method based on coarse and fine granular features, accurately locate and extract the best frame and its preceding and following frames. Frame images, ensuring that the selected images are representative and informative; Step 6.2.1 defines coarse-grained and fine-grained features. Since the inspection robot remains in a fixed position on the sliding track, while the various pig houses shown in the video are constantly changing, the coarse-grained feature can be defined as the frame number corresponding to when the fence is precisely positioned at a specified location, while the fine-grained feature refers to the specific pixel location features of the pig herd in that frame image. Variables can be defined. ,in This indicates the pixel location information of the pig herd in the image. This indicates the frame number corresponding to when the pig herd is precisely located in the video; Step 6.2.2, as follows Figure 2 As shown, each pigsty unit is equipped with a pair of numbered fences, labeled Fence S and Fence E respectively. These two fences are arranged in pairs to form a complete boundary. Furthermore, the current pigsty's fences... Fence with the next pigsty Consistency. Based on fisheye camera features, the identification and tracking model in step 5.6 is used to identify the first complete appearance of the fence S in a single pigsty in the video data, and the number of frames corresponding to when the pixel position is consistent with the expected pixel position within a reasonable error range is recorded. Its reasonable error range is set based on the image resolution, that is, it does not exceed 1% of the image diagonal length. The calculation formula is as follows: (twenty two) in, This represents the pixel position information of the current target object. This represents the desired pixel position information. This indicates the acceptable error range.

[0031] Similarly, in step 6.2.3, when fence E corresponding to fence S first appears completely in step 6.2.2, record the number of frames when its pixel position is consistent with the expected pixel position within a reasonable error range. The calculation formula is as follows: (twenty three) in, This represents the pixel position information of the current target object. This indicates the desired pixel position. This indicates the acceptable error range.

[0032] Step 6.2.4: Calculate the optimal frame position based on the results obtained in steps 6.2.2 and 6.2.3. This refers to the number of frames in the exact center of the pigsty, calculated using the following formula: (twenty four) in, This indicates the number of frames representing the transition time of discarded data at the beginning of the video data phase. This part of the data cannot identify the entire process of the fence in a single pigsty from its first appearance to its disappearance. Step 6.2.5: Accurately locate the optimal frame data calculated in step 6.2.4. Extract the frame and the frames before and after it. Frame image data, obtained Frame image data. Based on the information content of the optimal frame, namely image entropy, total edge intensity, and number of feature points, The value is determined based on this characteristic: if the amount of information in the optimal frame data is low, then Take a larger value (e.g., 5, 6, 7, etc.); if the optimal frame has a high amount of data information, then... Take the smaller value (e.g., 2, 3, 4, etc.); Step 6.2.6, according to "the next single pig house Equal to the previous single pigsty The rule is to repeat steps 6.2.2 to 6.2.5 from the beginning to the end of the video data inspection until all pigpens in the pasture have been inspected, meaning the starting and ending pigpens numbers are the same. During this process, discarded data from the end of the video data inspection should be removed, meaning discarded data that cannot identify the entire process of a fence in a single pigpen from its first appearance to its disappearance.

[0033] Step 6.3: The image quality is quantitatively evaluated using the Feature Similarity Index Mersure (FSIM) assessment system to ensure that the selected images have sufficient quality for subsequent processing; Step 6.3.1: Regarding the images extracted in Step 6.2.5... For each frame of data, the phase consistency (PC) value is calculated using the following formula: (25) in, Indicates the first The amplitude of each Fourier component, express Local energy function It is a small positive number; to avoid the denominator being 0... It can be represented as: (26) in, and Discrete signals Each of the subvariables is used to calculate the reference image and the distorted image at position x. and Next, PC similarity was further calculated. ,in, This is a constant used to prevent the denominator from being zero. (27) Step 6.3.2, next, calculate the gradient feature extraction value (Gradient Magnitude, GM) for each image using the Scharr operator, as shown below. and These are the gradient values ​​in the horizontal and vertical directions, respectively. The two matrices are in Scharr operator representation form. The gradient is calculated. and Then, the final gradient value is obtained. : (28) (29) Calculate the reference image and the distorted image at position x. and Next, the GM similarity was calculated. ,in, It is a constant used to enhance stability: (30) Step 6.3.3: The PC similarity from step 6.3.1 and the GM similarity from step 6.3.2 are fused. The calculation formula is as follows: The comprehensive local feature similarity at position x. To control the parameters of PC weights, To control the parameters of GM weights, All values ​​are set to 1 by default. (31) Step 6.3.4: Calculate the structural similarity index (FSIM) for each image, expressed as follows: (32) in, It can be represented as: (33) Finally, the FSIM values ​​of each image are compared, and the frame image with the highest FSIM value is selected to obtain the optimal image.

[0034] Step 6.4 employs a multi-step feature enhancement image quality optimization method, such as... Figure 5 The diagram shows the framework of this method, which includes image registration technology, Laplacian edge detection algorithm, and histogram equalization method to optimize the optimal image quality and target object position selected in step 6.3, ensuring that the expected standards are met.

[0035] Step 6.4.1: Use image registration technology to precisely adjust the position of the target object in the optimal image obtained in step 6.3.4 to ensure that the position of the target object in the optimal image meets the expected standard and achieves the ideal alignment effect. Step 6.4.1.1: Detect representative location points in the image to be registered (also known as the floating image), including the edges of closed regions, contour lines, and vertices at the intersection of boundaries, such as the feature points, vertices, or contour inflection points of the target object's fence structure mentioned in Step 6.2. These location points are collectively referred to as control points (CPs). Step 6.4.1.2: Based on the control points extracted in Step 6.4.1.1, a correspondence between control point pairs is established between the floating image and the reference image (also known as the fixed image), i.e., a feature matching relationship. By extracting the local feature descriptors of each control point, and based on the similarity measure between descriptors, combined with the spatial distribution consistency between control points, a set of point pairs with high matching confidence is selected as the basis for the subsequent geometric transformation estimation in Step 6.4.1.3; Step 6.4.1.3, based on the feature matching in step 6.4.1.2, uses the robust estimation method RANSAC to remove erroneous matching point pairs, and further selects a geometric transformation model type (such as rigid transformation, affine transformation, or perspective transformation) suitable for the current image registration scenario. By performing least-squares fitting on the retained high-confidence point pairs, the parameters of the transformation matrix are estimated to maximize the structural consistency and local geometric feature alignment between the floating image and the fixed image. Step 6.4.1.4: Based on the geometric transformation model estimated in Step 6.4.1.3, the original coordinates of all pixels in the floating image are mapped to the coordinate system of the fixed image using a transformation matrix. The mapped image is then resampled using an interpolation method to generate a new image spatially aligned with the fixed image. After the geometric correction process, the floating image and the fixed image achieve precise overlap in structural position, completing the registration.

[0036] Step 6.4.2 uses the Laplacian edge detection algorithm to extract edge information from the image, providing key features for further analysis and target recognition. The Laplacian operation is defined as follows: (34) This algorithm works by using the center pixel in the local neighborhood of the image. For reference, the second derivative approximation (i.e., Laplacian value) of the above 4 neighborhood templates is calculated, and edge detection is performed based on the gradient calculation result, thereby significantly highlighting edge pixels in the image; Step 6.4.3: Histogram equalization is used to adjust the image illumination information to enhance the image contrast and brightness uniformity, thereby improving image quality and enhancing subsequent processing effects.

[0037] Step 6.4.3.1 involves scanning the original grayscale image pixel by pixel and counting the frequency of each grayscale level to construct a complete grayscale histogram. ,in Indicates grayscale level. This represents the total number of gray levels. Step 6.4.3.2, based on the histogram obtained in step 6.4.3.1. The corresponding discrete form of the cumulative distribution function (CDF) is calculated, and its mapping method is expressed as follows: (35) in, grayscale The cumulative distribution value, It is the total number of pixels in the image. It is grayscale. The number of pixels, It is a grayscale index value; Step 6.4.3.3, using the cumulative distribution function from step 6.4.3.2. Derive the mapping relationship between input gray levels and output gray levels, and assign each input gray level... Mapped to output grayscale levels The transformation formula is as follows: (36) in, The original grayscale value. For the equalized grayscale values, The maximum grayscale value of the image. This indicates a rounding operation. This mapping ensures that low-frequency gray values ​​in the original image are stretched to a wider gray range, while high-frequency gray values ​​are compressed to a narrower gray range, thereby balancing the overall gray distribution and improving image contrast. Step 6.4.3.4: Perform grayscale transformation on the original image according to the mapping relationship in step 6.4.3.3, that is, read the original grayscale value of each pixel in the image. And replace it with the corresponding equalized grayscale value by looking up a table. This grayscale transformation operation is applied to the entire image, ultimately producing an image with a more balanced brightness distribution and clearer details.

[0038] Example 8 The specific process of step 7 is as follows: Step 7.1, obtain the optimal image data processed in step 6 to provide high-quality input for subsequent training and testing; Step 7.2: Based on the learning task in Step 4.4, adjust the data configuration file and training parameters, train the model using the model in Step 5, and export the trained model for further use. Step 7.3: Select the best weight file and test parameters, and evaluate the optimal image data selected in Step 7.1 to verify the performance of the model in a real pigsty scenario. Step 7.4: Evaluate the final test data and save the test results for subsequent analysis and model optimization.

[0039] like Figure 6 The image shown is the result of the YOLOV8 and Deepsort fusion optimization model in the optimal image of the pigsty. Under the special shooting conditions of the pasture, the method of the present invention performs well in identifying dynamic objects, especially in the optimal image part after using the optimal image localization and extraction method based on coarse and fine granular features, which has more accurate and realistic results.

[0040] Example 9 To verify the optimization effect of the improvements on the model, this invention conducted four ablation experiments on the validation set divided in step 3.3 without changing the experimental environment. The results of the ablation experiments are shown in Table 1. A represents replacing the original YoloV8 detection head module with the PP-YOLOEHead lightweight detection head; B represents introducing the Temporal Shift Module at the end of the Backbone based on A; C represents adopting a hybrid activation function strategy of LeakyReLU and SiLU based on B.

[0041] Table 1 shows the performance comparison of the improved target recognition and tracking model on the same dataset after performance enhancements. Experimental results show that after replacing the original detection head with the PP-YOLOE Head, although the detection accuracy mAP decreased by 0.8%, the inference speed increased by 15 FPS, and the model size decreased by 23 MB. On this basis, after introducing the Temporal ShiftModule, the detection accuracy mAP increased by 2.1%, the tracking continuity IDF1 increased by 3.2%, and the model computation increased by only 0.8 GFLOPs. After further adopting the hybrid activation function strategy, the detection accuracy mAP increased by another 1.3%, and the robustness of the model in motion blur scenarios was improved by 18%. Through the above improvements, the detection accuracy mAP was improved by 3.4%, the tracking continuity IDF1 increased by 4.5%, and the model inference speed increased by 12 FPS, while keeping the computational cost basically unchanged.

[0042] Table 1

[0043] The experimental environment used Microsoft Windows 10 Professional operating system, an 11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz 8-core 16-logic processor, an NVIDIA GeForce RTX 4090 24GB VRAM graphics card, 32GB RAM, PyTorch 1.11.0 deep learning framework, CUDA 11.3 GPU acceleration library, TorchVision 0.12.0, Python programming language, Python 3.8.20 interpreter. During training, the input image size was 640x640, each model had 100 training cycles, a batch size of 32, and a multi-threaded data loading count of 16.

Claims

1. A target recognition and tracking method based on dynamic scene perception of inspection robots, characterized in that: Specifically, the steps include the following: Step 1: Build an automatic pigsty inspection system and collect inspection video data to obtain real pigsty video data; Step 2: Preprocess the pigs-ywuzm public dataset; Step 3: Preprocess the real pigsty video data; Step 4: Construct a multi-source ensemble dataset and perform augmentation processing on the multi-source ensemble dataset; Step 5: Build the identification and tracking model; Step 6: Process the real pigsty video data obtained in Step 1; Step 7: Use the enhanced dataset from Step 4 to train the recognition and tracking model constructed in Step 5, and use the dataset processed in Step 6 to test the trained model.

2. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 1.1: Construct a closed loop track along the ceiling area above adjacent pens in the pigsty, covering multiple pig pens and activity areas arranged sequentially from pen 1 to pen m. Step 1.2: Integrate an image data acquisition terminal onto the inspection robot body; Step 1.3: Fix the inspection robot with integrated image data acquisition terminal on the circular track built in Step 1.1, and set the default cruising route to a clockwise direction along the track. Step 1.4: During the robot inspection process, the visual acquisition module in the image data acquisition terminal continuously collects videos of the pig herd's behavior throughout the day, receives and stores the inspection video data in real time, and marks the data according to the location of the pigsty to form real pigsty video data covering different time periods and locations.

3. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 2, characterized in that: The specific process of step 2 is as follows: Step 2.1: Obtain the pigs-ywuzm public dataset from the Roboflow platform; Step 2.2: Adjust the image sizes in the public dataset to obtain a standard-sized image dataset; Step 2.3: Perform independent normalization on the color channels of each image in the standard-size image dataset processed in Step 2.2 to obtain the preprocessed pigs-ywuzm public dataset.

4. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 3, characterized in that: The specific process of step 3 is as follows: Step 3.1: Obtain the real pigsty video data obtained in Step 1.4; Step 3.2: Use interpolation to count the number of frames in the target image data from the real pigsty video data obtained in Step 3.

1. Perform calculations, according to Image segmentation was performed on real pigsty video data to extract valid frame image data. The interpolation calculation formula is as follows: （1） in, , , To the number of images to be cut, These are real pigsty video data from various time periods, and Corresponding to the start time and start frame in the video. Corresponding to the end time and end frame in the video; Step 3.3, divide the dataset from Step 3.2 into portions according to an 8:1:1 ratio. The dataset was divided into training, validation, and test sets. The image feature labeling tool LabelImg was used to label the training and validation sets to obtain a real pigsty dataset.

5. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 4, characterized in that: The specific process of step 4 is as follows: Step 4.1: Apply an adaptive improved Gaussian filtering algorithm to denoise the pigs-ywuzm public dataset processed in Step 2.3 and the real pigsty dataset processed in Step 3.3, respectively. Step 4.2: Integrate the public dataset of pigs-ywuzm processed in step 4.1 with the real pig house dataset to obtain a multi-source integrated dataset, and then perform data cleaning on the multi-source integrated dataset; Step 4.3: The multi-source ensemble dataset obtained in Step 4.2 is processed using the Mosaic data augmentation method; Step 4.4: Apply a multi-task learning method based on behavior and scene features to set learning tasks for the multi-source ensemble dataset processed in Step 4.

3.

6. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 5, characterized in that: The specific process of step 4.4 is as follows: Step 4.4.1: Perform an action learning task on dynamic objects using the pigs-ywuzm public dataset processed in Step 4.3; Step 4.4.2: Perform a scene learning task for dynamic objects on the real pigsty dataset processed in Step 4.3; Step 4.4.3 involves adaptively assigning weights to the learning tasks from steps 4.4.1 and 4.4.2, and then performing a comprehensive modeling of the two types of tasks to obtain the comprehensive task value. The calculation formula is as follows: （2） in, and This represents the task value and weight coefficient in action, behavior, and state learning tasks. and This represents the task value and weight coefficient in learning tasks within real-life and video scenarios. This represents the moderating term for other potential influencing factors.

7. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 6, characterized in that: The specific process of step 5 is as follows: Step 5.1: Initialize all modules of the YOLOV8 target recognition model; Step 5.2, adjust the framework structure in the YOLOV8 model, including replacing the detection head structure and introducing a temporal modeling module; Step 5.3: Based on step 5.2, the Leaky ReLU activation function is used instead of the default SiLU activation function in the convolutional layers of the YoloV8 model, while the Sigmoid activation function is retained in the output layer. Step 5.4: The improved model from Step 5.3 is enhanced by combining the Motion Blur and Dynamic Soft Labeling strategies. Step 5.5: Initialize the DeepSort tracking module; Step 5.6: Define and instantiate the DeepSort tracking module class object in the YOLOV8 recognition model after the processing in step 5.

4. After each frame of image is recognized by YOLOV8, the detected bounding box and class confidence are passed as input to the DeepSort module to obtain the recognition and tracking model.

8. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 7, characterized in that: The specific process of step 5.2 is as follows: Step 5.2.1: In the Head part of the YOLOV8 model, the original detection head module of YOLOV8 is replaced with the PP-YOLOE Head lightweight detection structure; Step 5.2.2: A lightweight temporal modeling module (TSM) is introduced at the backbone end of the YOLOV8 model to enhance the 2D convolutional network by processing the feature maps of the frame data through Temporal Shift.

9. The target recognition and tracking method based on dynamic scene perception of inspection robots according to claim 7, characterized in that: The specific process of step 5.3 is as follows: Step 5.3.1: In the shallow feature extraction stage of the Backbone, the original activation function is replaced with the Leaky ReLU activation function, that is, Leaky ReLU is applied after each convolutional module. The calculation formula is as follows: （3） in, Represented as the leakage coefficient, For input values; Step 5.3.2, in the deep feature extraction stage of the backbone, uses the SiLU activation function, which is a function defined by the following formula immediately following each linear transformation output: （4） in, For input values, This is represented by the Sigmoid function, with an output range of (0, 1). Step 5.3.3: In the output layer of the detector head, the Sigmoid activation function is used to perform independent binary classification for each category. That is, the existence probability of each category is obtained by applying the following formula to each element of the category vector output by each predicted box. : （5） in, For input values; Step 5.3.4, uniformly set the final output structure as follows ,in For batch size, For the number of prediction boxes, 5 represents the number of categories. , respectively represent the center coordinates of the target box. Width and height And the confidence level of the target. .

10. The target recognition and tracking method based on dynamic scene perception of an inspection robot according to claim 9, characterized in that: The specific process of step 6 is as follows: Step 6.1: Obtain the real pigsty video data obtained in Step 1.4; Step 6.2: Using an optimal image localization and extraction method based on coarse and fine granular features, accurately locate and extract the best frame and the frames before and after the best frame from the real pigsty dataset. Frame image; Step 6.3: Quantitatively evaluate the image quality from Step 6.2 using the feature similarity index evaluation system; Step 6.4: The optimal image quality and target object position selected in Step 6.3 are optimized using a multi-step feature enhancement image quality optimization method, and the optimized optimal image is output.

Images