Dynamic target tracking method and device based on deep learning and storage medium

Abstract

The invention discloses a dynamic target tracking method and device based on deep learning and a storage medium, and the method comprises the steps: inputting an image sequence set into a target recognition model for feature extraction, and obtaining target features corresponding to each image in the image sequence set; the target identification model is obtained by improving a YOLOv8 model, and the YOLOv8 model comprises a backbone network, a neck network and a head network which are connected in sequence; replacing the C2f module of the eighth layer in the backbone network with an improved C2f module, and replacing the neck network with an improved neck network to obtain a target identification model; determining a target track of the target object based on the target features corresponding to the images in the image sequence set; according to the method, feature extraction can be effectively performed on the image sequence set of the target object in the target area through the target recognition model obtained through improvement based on the YOLOv8 model, so that the target trajectory is generated, and high-precision recognition tracking of the target object is realized under the condition that the recognition efficiency is not influenced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a dynamic target tracking method and device based on deep learning and a storage medium. BACKGROUND

[0002] With the rapid development of the power system, the transmission line inspection technology has undergone a transformation from manual inspection to automation and intelligence. Early manual inspection is low in efficiency, high in labor intensity and has safety hazards. The inspection personnel need to climb the tower, walk through complex terrain, which not only consumes time and effort, but also faces the risk of falling from a high altitude, electric shock, etc. In recent years, the application of new technologies such as unmanned aerial vehicles and intelligent sensors has promoted the automation process of inspection - unmanned aerial vehicles can quickly cover areas that are difficult for humans to reach, and intelligent sensors can monitor equipment status in real time, greatly improving the efficiency and safety of inspection.

[0003] In this intelligent process, the importance of target tracking technology is increasingly prominent. It is not only a key support for efficient inspection, but also a core link to ensure data accuracy and operation safety. For example, when inspecting by unmanned aerial vehicle, the target tracking needs to stabilize and lock the key components such as towers and wires to ensure the continuity and clarity of image acquisition. However, the existing target tracking methods currently face double challenges in the actual transmission line inspection scene: first, the severe changes in light (such as strong light reflection, low contrast in rain) and complex occlusions (such as trees, construction machinery) will destroy the continuity of target features, leading to blurred segmentation boundaries, tracking frame shifts, and even target loss, significantly reducing recognition accuracy; second, in order to ensure recognition accuracy, the existing target tracking methods need to improve the complexity of the recognition model, resulting in an increase in the number of model parameters, serious waste of computing resources, and difficulty in meeting real-time requirements of feature extraction efficiency on embedded devices, forming a "precision-efficiency" dilemma. SUMMARY

[0004] The purpose of the present application is to provide a dynamic target tracking method and device based on deep learning and a storage medium, which can extract features of a target object in a target region by improving a target recognition model based on a YOLOv8 model to generate a target trajectory, thereby realizing high-precision recognition and tracking of the target object without affecting recognition efficiency.

[0005] To achieve the above purpose, the present application adopts the following technical solutions: In a first aspect, the present application provides a dynamic target tracking method based on deep learning, comprising: obtaining an image sequence set of a target object in a target region; The image sequence set is input into the target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set; the target recognition model is an improvement on the YOLOv8 model, which includes a backbone network, a neck network and a head network connected in sequence; the C2f module of the eighth layer in the backbone network is replaced with an improved C2f module, and the neck network is replaced with an improved neck network to obtain the target recognition model; The target trajectory of the target object is determined based on the target features corresponding to each image in the image sequence set.

[0006] Optionally, the process of acquiring the target recognition model includes: Obtain historical image sequences and annotation information of the target object in the target region; The historical image sequence set and annotation information are used as the training set and input into the preset recognition model for training to obtain the optimal parameters of the preset recognition model; Based on the optimal parameters of the preset recognition model, the target recognition model is obtained.

[0007] Optionally, the image sequence is input into a target recognition model for feature extraction to obtain target features corresponding to each image in the image sequence set, including: The images in the image sequence set are input into the target recognition model one by one as target images for feature extraction, so as to obtain target features that correspond one-to-one with the target images; wherein, the target recognition model also needs to be updated in stages during the process of inputting the image sequence into the target recognition model for feature extraction.

[0008] Optionally, the steps for periodically updating the target recognition model include: Obtain the target background and standard background of the image sequence set; The foreground image is obtained by segmenting the target image based on the target background; The foreground image is judged as foreground. If the foreground image is judged as foreground within a preset number of times, the foreground image is marked as a false background; otherwise, the foreground image is not marked as a false background. The preset number of times is the number of times the target recognition model processes the images in the image sequence set in sequence. The similarity between the pseudo background and the standard background is calculated. If the similarity is greater than the similarity threshold, the pseudo background is updated as the standard background; otherwise, the standard background is not updated. Images containing pseudo-background regions are extracted from the image sequence set as negative sample datasets, and images containing standard background regions are extracted as positive sample datasets. The negative sample dataset and the positive sample dataset are then combined to obtain the incremental training set. The incremental training set is input into the target recognition model for training, and the parameters of the target recognition model are fine-tuned to obtain a target recognition model that is updated in stages.

[0009] Optionally, the process of obtaining the target background and standard background of the image sequence set includes: Randomly select pixels from the images in the image sequence set to obtain the initial background; Calculate the expected value and standard deviation of pixels in the image sequence set, and obtain the pixel threshold range based on the expected value and standard deviation; The pixels in the initial background are judged for anomalies. If a pixel in the initial background belongs to the pixel threshold range, it is judged as a normal pixel. Otherwise, it is judged as an abnormal pixel and the abnormal pixel is replaced to obtain the target background. The target background is updated by eight neighborhoods when the number of times the target recognition model processes the images in the image sequence set meets the preset number of times.

[0010] Based on the standard deviation of pixels, pixel selection is performed on images in the image sequence set to obtain a standard background.

[0011] Optionally, the improved C2f module is obtained by replacing the Conv module of the C2f module in the YOLOv8 model with a dynamic snake convolution module.

[0012] Optionally, the improved neck network includes: a first convolutional module, a second convolutional module, and a third convolutional module that are respectively connected to the fourth layer C2f module, the sixth layer C2f module, and the ninth layer SPPF module of the target recognition model, as well as an upsampling module and a downsampling module connected in sequence. The upsampling module includes a first upsampling unit, a second upsampling unit, and a third upsampling unit connected in sequence. Each of the first, second, and third upsampling units includes a Concat module and a C2f module connected in sequence. Specifically, the Concat module of the first upsampling unit concatenates the outputs of the second and third convolutional modules; the Concat module of the second upsampling unit concatenates the outputs of the sixth-layer C2f module, the first convolutional module, and the first upsampling unit; and the Concat module of the third upsampling unit concatenates the outputs of the fourth-layer C2f module and the second upsampling unit. The downsampling module includes a Conv module and a first downsampling unit and a second downsampling unit connected in sequence. Both the first downsampling unit and the second downsampling unit include a Conv module, a Concat module, and a C2f layer connected in sequence. The Concat module in the first downsampling unit is used to concatenate the output results of the Conv module in the first downsampling unit and the C2f module in the second upsampling unit. The Concat module in the second downsampling unit is used to concatenate the output result of the Conv module in the second downsampling unit and the output result of the C2f module in the second upsampling unit that has been processed by the Conv module of the downsampling module.

[0013] Optionally, the operation of the improved neck network includes: The outputs of the ninth-layer SPPF module, the sixth-layer C2f module, and the fourth-layer C2f module in the backbone network are used as inputs to the improved neck structure; the output of the ninth-layer SPPF module is used as input three, the output of the sixth-layer C2f module is used as input two, and the output of the sixth-layer C2f module is used as input one. Input 1, Input 2, and Input 3 are respectively input into the first convolution module, the second convolution module, and the third convolution module for convolution operations to obtain the first feature map, the second feature map, and the third feature map; the second feature map and the third feature map are input into the Concat module of the first upsampling unit for concatenation to obtain the fourth feature map; the fourth feature map is input into the C2f module of the first upsampling unit to obtain the fifth feature map; Input 2, the first feature map, and the fifth feature map are input into the Concat module of the second upsampling unit for concatenation to obtain the sixth feature map; the sixth feature map is input into the C2f module of the second upsampling unit to obtain the seventh feature map; the seventh feature map is concatenated with the first input into the Concat module of the third upsampling unit to obtain the eighth feature map; the eighth feature map is input into the C2f module of the third upsampling unit to obtain the ninth feature map; the ninth feature map is used as the input of the third detection head; The ninth feature map is input into the Conv module of the first downsampling unit to obtain the tenth feature map; the tenth feature map and the seventh feature map are input into the Concat module of the first downsampling unit and concatenated to obtain the eleventh feature map; the eleventh feature map is input into the C2f module of the first downsampling unit to obtain the twelfth feature map; the twelfth feature map is used as the input of the second detection head; The twelfth feature map is input into the Conv module of the first downsampling unit to obtain the thirteenth feature map; the seventh feature map is input into the Conv module of the downsampling module to obtain the fourteenth feature map; the thirteenth and fourteenth feature maps are input into the Concat module of the second downsampling unit and concatenated to obtain the fifteenth feature map; the fifteenth feature map is input into the C2f module of the second downsampling unit to obtain the sixteenth feature map, and the sixteenth feature map is used as the input of the first detection head.

[0014] Secondly, the present invention provides a deep learning-based dynamic target tracking device, comprising: The data acquisition module is used to acquire a set of image sequences of target objects in the target area; The pre-update module is used to update the preset recognition model based on the image sequence set of the target object to obtain the target recognition model; The feature recognition module is used to input the image sequence set into the target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set. The target recognition model is an improvement on the YOLOv8 model, which includes a backbone network, a neck network, and a head network connected in sequence. The C2f module in the eighth layer of the backbone network is replaced with an improved C2f module, and the neck network is replaced with an improved neck network to obtain the target recognition model.

[0015] The trajectory determination module is used to determine the target trajectory of the target object based on the target features corresponding to each image in the image sequence set.

[0016] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed, implements the deep learning-based dynamic target tracking method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention provides a deep learning-based dynamic target tracking method, device, and storage medium. The method inputs a set of image sequences of the target object into a target recognition model improved from the YOLOv8 model, obtaining the target features corresponding to each image in the image sequence set, thereby obtaining the target trajectory of the target object. Specifically, the target recognition model replaces the C2f module in the eighth layer of the YOLOv8 model backbone network with an improved C2f module and replaces the neck network with an improved neck network. This addresses the impact of lighting changes and occlusion in real-world scenarios on the accuracy of segmentation, as well as the problem of low feature extraction efficiency due to excessive computational power consumption during target recognition, thus ensuring the accuracy of defect identification in subsequent power transmission line inspections.

[0018] This invention provides a deep learning-based dynamic target tracking method. Compared with the YOLOv8 model, the target recognition model proposed in this invention replaces the Conv module in the C2f module of the YOLOv8 model with a dynamic snake convolution module, which can more flexibly capture complex features in images and improve the accuracy and robustness of feature extraction.

[0019] This invention provides a deep learning-based dynamic target tracking method. Compared to the YOLOv8 model, the target recognition model proposed in this invention improves the neck network of the YOLOv8 model, enabling the model to better utilize multi-scale features for target recognition, thereby improving the accuracy and recall of the recognition.

[0020] This invention provides a deep learning-based dynamic target tracking method. In the process of feature extraction from the image sequence input target recognition model, the target recognition model also needs to be updated in stages to eliminate the misjudgment of background changes as foreground targets due to occlusion or lighting (false detection), or the missed detection of targets due to similar background and target features, thereby increasing the accuracy of target tracking. Attached Figure Description

[0021] Figure 1 The diagram shown is a flowchart of a deep learning-based dynamic target tracking method in one embodiment of the present invention. Figure 2 The diagram shown is a partial structural schematic of a target recognition model in one embodiment of the present invention; Figure 3 The diagram shown is a schematic representation of a process for periodically updating a target recognition model in one embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings. The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and B can represent: A alone, A and B simultaneously, and B alone. In addition, the descriptions involving "first," "second," etc. in the present invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, features defined with "first" or "second" can explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0023] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention. Example 1

[0024] like Figure 1 As shown in the figure, this embodiment of the invention introduces a dynamic target tracking method based on deep learning, including the following steps: S1: Obtain the image sequence set of the target object in the target region; S2: Input the image sequence set into the target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set; the target recognition model is an improvement on the YOLOv8 model, which includes a backbone network, a neck network and a head network connected in sequence; replace the C2f module of the eighth layer in the backbone network with the improved C2f module, and replace the neck network with the improved neck network to obtain the target recognition model.

[0025] S3: Determine the target trajectory of the target object based on the target features corresponding to each image in the image sequence set.

[0026] This embodiment introduces a deep learning-based dynamic target tracking method, which can extract features from image sequences of target objects in the target region by improving the target recognition model based on the YOLOv8 model, thereby generating target trajectories and achieving high-precision recognition and tracking of target objects without affecting recognition efficiency.

[0027] In this embodiment, the image sequence set of the target object in step S1 is obtained by periodically acquiring images of the target object in the target area using a camera device; periodic image acquisition can completely record the dynamic changes of the target object in the target area and avoid missing key information due to excessively long acquisition intervals.

[0028] In this embodiment, the process of acquiring the target recognition model in step S2 includes: Obtain historical image sequences and annotation information of the target object in the target region; The historical image sequence set and annotation information are used as the training set and input into the preset recognition model for training to obtain the optimal parameters of the preset recognition model; Based on the optimal parameters of the preset recognition model, the target recognition model is obtained.

[0029] Specifically, image sequences containing target objects are collected from the target area (specific environment) to ensure that the data covers different angles, lighting, occlusion and other scenes to obtain a historical image sequence set. Then, the target objects in the images are labeled using annotation tools (such as LabelImg, CVAT) to generate a format (annotation information) supported by the preset recognition model. Subsequently, a weighted combination of classification loss (BCE), bounding box loss (CIoU) and targetability loss (BCE) is used to calculate the comprehensive loss, further optimize the parameters of the preset recognition model, and finally obtain the target recognition model.

[0030] Specifically, before using historical image sequences and annotation information as training sets and inputting them into the preset recognition model for training, operations such as rotation, flipping, and color transformation are performed on the images in the historical image sequence sets to improve the model's generalization ability.

[0031] In this embodiment, the target recognition model is obtained by improving the YOLOv8 model. Specifically, the YOLOv8 model includes a backbone network, a neck network, and a head network. Since the YOLOv8 model is existing technology, the specific model structure will not be described in detail here. like Figure 2 The diagram shows a partial structure of the target recognition model, corresponding to the backbone and neck network of the YOLOv8 model. The improvements include: Replace the C2f module in the backbone network of the YOLOv8 model with the improved C2f module; The improved C2f module is obtained by replacing the Conv module of the original YOLOv8 model's C2f module with a dynamic snake convolution module. The C2f module includes a sequentially connected Conv module, Split module, multiple Bottleneck modules, Concat module, and Conv module. The working process of the C2f module is as follows: the input feature map is processed by the Conv module to obtain an intermediate feature map, and then split into two parts by the Split module. One part is input into multiple Bottleneck modules for processing to obtain the output results of multiple Bottleneck modules. The other part is concatenated with the output results of multiple Bottleneck modules by the Concat module. Finally, it is processed by the Conv module to obtain the output result of the C2f module.

[0032] The Conv module consists of a sequentially connected Conv2d layer, a BatchNorm2d layer, and SILU. The Bottleneck module is the bottleneck block, the structure of which is known to those skilled in the art and will not be described in detail here. The improved C2f module replaces the first and last Conv modules in the C2f module with a Dynamic Snake Convolution (DSC) module, which can more flexibly capture complex features in the image. The Dynamic Snake Convolution module can dynamically adjust the shape of the convolution kernel according to the input features, thereby better adapting to the shape changes of the target, enhancing the model's ability to detect targets of different scales and shapes, and improving the accuracy and robustness of feature extraction.

[0033] Improve the neck network of the YOLOv8 model: The improved neck network includes: a first convolutional module (Conv1), a second convolutional module (Conv2), a third convolutional module (Conv3), and sequentially connected upsampling and downsampling modules; the first convolutional module is connected to the fourth layer C2f module of the target recognition model; the second convolutional module is connected to the sixth layer C2f module of the target recognition model; the third convolutional module is connected to the ninth layer SPPF module of the target recognition model; wherein, the first, second, and third convolutional modules are all Conv modules (sequentially connected convd layers, BatchNorm2d layers, and SiLU activation functions), with the same structure as the Conv modules in the upsampling and downsampling modules.

[0034] Specifically, the upsampling module includes a first upsampling unit, a second upsampling unit, and a third upsampling unit connected in sequence; each of the first, second, and third upsampling units includes a Concat module and a C2f module connected in sequence; wherein, the Concat module of the first upsampling unit is used to concatenate the outputs of the second and third convolutional modules; the Concat module of the second upsampling unit is used to concatenate the outputs of the sixth-layer C2f module, the first convolutional module, and the first upsampling unit; the Concat module of the third upsampling unit is used to concatenate the outputs of the fourth-layer C2f module and the second upsampling unit. Specifically, the downsampling module includes a Conv module and a first downsampling unit and a second downsampling unit connected in sequence; both the first and second downsampling units include a Conv module, a Concat module, and a C2f layer connected in sequence; wherein, the Concat module in the first downsampling unit is used to concatenate the output results of the Conv module in the first downsampling unit and the C2f module in the second upsampling unit; the Concat module in the second downsampling unit is used to concatenate the output result of the Conv module in the second downsampling unit and the output result of the C2f module in the second upsampling unit processed by the Conv module of the downsampling module.

[0035] Based on the improved neck network structure described above, its working process will be described in detail below: The improved neck network operates as follows: The outputs of the ninth-layer SPPF module, the sixth-layer C2f module, and the fourth-layer C2f module in the backbone network are used as inputs to the improved neck structure; the output of the ninth-layer SPPF module is used as input three, the output of the sixth-layer C2f module is used as input two, and the output of the sixth-layer C2f module is used as input one. Input 1, Input 2, and Input 3 are respectively input into the first convolution module, the second convolution module, and the third convolution module to perform convolution operations to obtain the first feature map, the second feature map, and the third feature map; The second feature map and the third feature map are concatenated in the Concat module of the first upsampling unit to obtain the fourth feature map; the fourth feature map is then input into the C2f module of the first upsampling unit to obtain the fifth feature map. The second feature map, the first feature map, and the fifth feature map are concatenated in the Concat module of the second upsampling unit to obtain the sixth feature map; the sixth feature map is then input into the C2f module of the second upsampling unit to obtain the seventh feature map. The seventh feature map is concatenated with the Concat function of the input-input third upsampling unit to obtain the eighth feature map; the eighth feature map is input to the C2f module of the third upsampling unit to obtain the ninth feature map; the ninth feature map is used as the input of the third detection head (T3). This layer-by-layer concatenation and processing method further strengthens the correlation between features, making the features continuously richer and more refined during the transmission process, providing high-quality feature input for the third detection head, and improving the detection capability of small targets or targets in complex scenes.

[0036] The ninth feature map is input into the Conv module of the first downsampling unit to obtain the tenth feature map; the tenth feature map and the seventh feature map are input into the Concat module of the first downsampling unit and concatenated to obtain the eleventh feature map; the eleventh feature map is input into the C2f module of the first downsampling unit to obtain the twelfth feature map; the twelfth feature map is used as the input of the second detection head (T2). This process further optimizes the hierarchical structure of the features through the re-convolution and concatenation of the features, enabling the second detection head to better utilize multi-scale features for target detection, thereby improving the model's detection accuracy and robustness for medium-scale targets.

[0037] The 12th feature map is input into the Conv module of the first downsampling unit to obtain the 13th feature map; the 7th feature map is input into the Conv module of the downsampling unit to obtain the 14th feature map; the 13th and 14th feature maps are then concatenated into the Concat module of the second downsampling unit to obtain the 15th feature map; the 15th feature map is then input into the C2f module of the second downsampling unit to obtain the 16th feature map, which is used as the input (T1) of the first detection head. This multi-path feature fusion method can fully utilize feature information at different levels, further enhancing the diversity and complementarity of features. Therefore, the first detection head can more effectively handle large-scale targets, improving the detection performance of the target recognition model for large targets, while also enhancing the overall generalization ability of the target recognition model.

[0038] Specifically, the first detection head, the second detection head, and the third detection head are the head network in the YOLOv8 model, i.e., the Head part, which is a structure known to those skilled in the art and will not be described in detail here; In this embodiment, the neck network of the YOLOv8 model is replaced with an improved neck network, further optimizing the feature fusion process. The improved neck network can more efficiently integrate feature information from different levels, reduce feature loss, and enhance the interaction between features. This improvement enables the model to better utilize multi-scale features for object detection, improving the accuracy and recall of recognition, especially when dealing with small targets and complex scenes, significantly improving the overall performance of the model.

[0039] In this embodiment, step S2 involves inputting the image sequence into a target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set, including: Images in the image sequence set are sequentially input into a target recognition model for feature extraction, resulting in target features corresponding one-to-one with each target image. During feature extraction from the image sequence, the target recognition model needs to be updated periodically. Specifically, during the entire process of feature extraction from all images in the image sequence set in chronological order, the target recognition model is updated when certain requirements are met, such as... Figure 3 As shown, the phased update steps are as follows: S01: Obtain the target background and standard background of the image sequence set; S02: Segment the target image based on the target background to obtain the foreground image; S03: Foreground image is judged. If the foreground image is judged as foreground within a preset number of times, the foreground image is marked as a false background. Otherwise, the foreground image is not marked as a false background. The preset number of times is the number of times the target recognition model processes the images in the image sequence set in sequence. Specifically, a counter N is defined to record the number of times the target recognition model processes N consecutive frames in an image sequence. A preset number of frames is set to 10, meaning the model needs to process 10 frames before completing one false background detection. If a region is identified as foreground in N consecutive frames, it is marked as false background. This helps distinguish between genuine foreground targets and misjudged parts caused by background changes, improving the accuracy and robustness of background segmentation and providing clearer foreground information for subsequent background updates and target recognition.

[0040] S04: Calculate the similarity between the pseudo background and the standard background. If the similarity is greater than the similarity threshold, then update the pseudo background as the standard background; otherwise, do not update the standard background. Specifically, the phase velocity threshold is set manually, and phase velocity calculation can calculate structural similarity (SSIM), mean squared error (MSE), or histogram similarity. This process dynamically adjusts the background model (an abstract representation of a static or slowly changing background in an image sequence) to better adapt to scene changes. When the pseudo-background has a high similarity to the standard background, it is used as the standard background for updating, which can effectively reduce background drift. If they are not similar, the pseudo-background is identified as a target feature, avoiding mistaking background changes for targets, thereby improving the accuracy and reliability of target recognition.

[0041] S05: Extract images containing pseudo-background regions from the image sequence set as negative sample datasets, extract images containing standard background regions as positive sample datasets, and combine the negative sample datasets and positive sample datasets to obtain the incremental training set; S06: Input the incremental training set into the target recognition model for training, fine-tune the target recognition model parameters, and obtain a target recognition model that is updated in stages.

[0042] Specifically, step S01, which involves acquiring the target background and standard background of the image sequence set, includes: S11: Randomly select pixels from the images in the image sequence set to obtain the initial background; S12: Calculate the expected value and standard deviation of pixels in the image sequence set, and obtain the pixel threshold range based on the expected value and standard deviation. Specifically, this process can effectively extract the statistical features of the background, providing a foundation for subsequent background optimization. By calculating the threshold range using the expected value and standard deviation, abnormal pixels can be accurately identified, thereby accurately distinguishing between normal and abnormal parts of the background, laying the foundation for background optimization.

[0043] S13: Perform anomaly detection on the pixels in the initial background. If a pixel in the initial background belongs to the pixel threshold range, it is determined to be a normal pixel; otherwise, it is determined to be an abnormal pixel, and the abnormal pixel is replaced to obtain the target background. S14: Based on the standard deviation of pixels, perform pixel selection on the images in the image sequence set to obtain a standard background.

[0044] Specifically, steps S13 and S14 optimize the accuracy of the background and reduce interference from noise and outliers. The generation of the target background and the standard background can more accurately reflect the real background state of the scene, providing high-quality background information for subsequent updates to the target recognition model, and improving the robustness and recognition accuracy of the model.

[0045] In this process, when the target recognition model processes the images in the image sequence a preset number of times, the target background also needs to undergo eight-neighborhood update. This eight-neighborhood update expands the pixel's neighborhood range (from the traditional 4-neighborhood to 8 directions), enabling more detailed adjustment of local background information, enhancing the background model's adaptability to complex scenes, reducing misjudgments caused by local background changes, and further improving the overall performance of background segmentation and target recognition. The technical content of eight-neighborhood update is well-known to those skilled in the art and will not be described in detail here.

[0046] Specifically, periodic updates to the target recognition model enable more accurate extraction of target object features, improving the accuracy and reliability of target recognition. Simultaneously, the application of target and standard backgrounds enhances the target recognition model's adaptability to complex scenes, further improving the performance and stability of target recognition.

[0047] In this embodiment, step S3 determines the target trajectory of the target object based on the target features corresponding to each image in the image sequence set, using the optical flow method. Its core is to extract the speed and direction information of the target by analyzing the motion patterns of pixels between consecutive frames, and then associate the same target in different frame images. The optical flow method is not an innovation of this invention and is known to those skilled in the art, so the process of determining the target trajectory will not be described in detail here.

[0048] This embodiment provides a deep learning-based dynamic target tracking method. It can extract features from image sequences of target objects in a target region using a target recognition model improved from the YOLOv8 model, thereby generating the target trajectory. For downstream tasks, namely automated inspection, it can also add dangerous action recognition and alarm processes, specifically including: By inputting the target trajectory into a deep learning model for dangerous action identification and issuing timely alerts when dangerous operations are detected, the safety monitoring capabilities of the target area are effectively improved. The deep learning model can intelligently analyze the target trajectory to identify potentially dangerous actions. This deep learning-based identification method has high accuracy and robustness, effectively avoiding false alarms and missed alarms. Once a dangerous operation is detected, an alarm is immediately triggered, promptly reminding relevant personnel to take measures to prevent accidents, thereby ensuring the safety of personnel and equipment within the target area and improving the overall safety management level. Specifically, the deep learning model can be a Temporal Convolutional Network (TCN), a Transformer model, etc.

[0049] In summary, the deep learning-based dynamic target tracking method provided by this invention ensures the comprehensiveness of the data and the accuracy of the model through periodic image acquisition and phased model updates; and then uses a target recognition model to extract features and determine the trajectory, thereby achieving accurate positioning and dynamic tracking of the target object.

[0050] Example 2

[0051] Based on Example 1, which introduces a target recognition model in a deep learning-based dynamic target tracking method, and YOLOv8 and YOLOv9, this example conducts comparative experiments on the following four types of datasets: The experimental setup is as follows: The model's training environment is based on the PyTorch 1.7.1 deep learning framework; the environment is Ubuntu 20.04, Python 3.7.11, CUDA=11.4, and the training graphics card configuration is NVIDIA RTX A6000 / 48G graphics card; the processor is Intel Xeon Platinum 8171M CPU@2.60GHz; the RAM is 96G; the local test computer uses an NVIDIA RTX3060Ti graphics card, an AMD Ryzen5 5600X processor, and 32G RAM. The four training datasets are shown in Table 1: Table 1 Content Parameter Table for Four Types of Datasets

[0052] The training parameters are shown in Table 2: Table 2 Training Parameters Parameter Name Parameter Description Parameter Value imgsz Input image size 1280×1280 batch Number of images per batch 8-16 epochs Training rounds 300-500 optimizer Optimization algorithm Adam lr0 Initial learning rate 0.001 box Bounding box loss weight 0.1 mosaic Mosaic data augmentation True degrees Random rotation angle 0-5° The target recognition model in this invention, as well as YOLOv8 and YOLOv9, were evaluated using three evaluation metrics: precision, recall, and accuracy, as shown in Tables 3, 4, and 5 below: Table 3 Evaluation Table of Target Recognition Model Object recognition model Sample Correct sample Incorrect sample Unrecognized sample Precision Recall Accuracy First class 225000 220300 75 4625 99.66 97.94 97.86 Second class 193300 188800 55 4445 99.71 97.66 97.68 Third class 215250 211300 85 3950 99.60 98.14 98.09 Fourth class 183250 179700 65 3550 99.64 98.06 98.02 Table 4 YOLOv8 Evaluation Table YOLOv8 Sample Correct sample Incorrect sample Unrecognized sample Precision Recall Accuracy First class 225000 213750 350 10900 98.38 95.16 94.78 Second class 193300 184000 400 8900 97.87 95.38 94.83 Third class 215250 205300 500 9450 97.63 95.60 95.19 Fourth class 183250 174200 450 8600 97.48 95.28 94.99 Table 5 YOLOv9 Evaluation Table YOLOv9 Sample Correct sample Incorrect sample Unrecognized sample Precision Recall Accuracy First class 225000 216000 300 8700 98.63 96.13 95.87 Second class 193300 186500 350 6450 98.16 96.66 96.12 Third class 215250 208500 400 6350 98.11 97.06 96.32 Fourth class 183250 176400 280 6570 98.44 96.39 96.20 In the table above, precision, recall, and accuracy are expressed as a percentage (%). Based on the evaluation table above, it can be seen that the tracking method of the present invention, based on the improved target recognition model of YOLOv8, has better accuracy in recognizing target features.

[0053] This embodiment also introduces a deep learning-based dynamic target tracking method based on Embodiment 1, and compares the accuracy of existing tracking methods (CTA, BOT SORT, Byte Track, OC SORT) through an experiment. The experimental results are shown in Table 6. Table 6 Comparison of different tracking methods

[0054] As can be seen from the table above, the tracking method of the present invention has a higher accuracy rate for each type of sample set than existing tracking methods, and the average accuracy rate is also much higher than that of existing tracking methods, thus effectively proving that the method of the present invention has better accuracy in precise positioning and dynamic tracking of target objects.

[0055] Example 3 This embodiment provides a deep learning-based dynamic target tracking method, including: The data acquisition module is used to acquire a set of image sequences of target objects in the target area; The pre-update module is used to update the preset recognition model based on the image sequence set of the target object to obtain the target recognition model; The feature recognition module is used to input the image sequence set into the target recognition model for feature extraction, thereby obtaining the target features corresponding to each image in the image sequence set; the target features correspond one-to-one with the images in the image sequence set; the target recognition model is obtained by improving the YOLOv8 model, which includes a backbone network, a neck network, and a head network connected in sequence; the C2f module in the eighth layer of the backbone network is replaced with an improved C2f module, and the neck network is replaced with an improved neck network to obtain the target recognition model.

[0056] The trajectory determination module is used to determine the target trajectory of the target object based on the target features corresponding to each image in the image sequence set.

[0057] Example 4 This embodiment provides a computer-readable storage medium storing a computer program that, when executed, implements the deep learning-based dynamic target tracking method described in Embodiment 1.

[0058] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A dynamic target tracking method based on deep learning, characterized in that, include: Obtain the image sequence set of the target object in the target region; The image sequence set is input into the target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set; the target recognition model is an improvement on the YOLOv8 model, which includes a backbone network, a neck network and a head network connected in sequence; the C2f module of the eighth layer in the backbone network is replaced with an improved C2f module, and the neck network is replaced with an improved neck network to obtain the target recognition model; The target trajectory of the target object is determined based on the target features corresponding to each image in the image sequence set.

2. The deep learning-based dynamic target tracking method according to claim 1, characterized in that, The process of acquiring the target recognition model includes: Obtain historical image sequences and annotation information of the target object in the target region; The historical image sequence set and annotation information are used as the training set and input into the preset recognition model for training to obtain the optimal parameters of the preset recognition model; Based on the optimal parameters of the preset recognition model, the target recognition model is obtained.

3. The deep learning-based dynamic target tracking method according to claim 3, characterized in that, The image sequence is input into the target recognition model for feature extraction to obtain the target features corresponding to each image in the image sequence set, including: The images in the image sequence set are input into the target recognition model one by one as target images for feature extraction, so as to obtain target features that correspond one-to-one with the target images; wherein, the target recognition model also needs to be updated in stages during the process of inputting the image sequence into the target recognition model for feature extraction.

4. The deep learning-based dynamic target tracking method according to claim 3, characterized in that, The steps for updating the target recognition model in stages include: Obtain the target background and standard background of the image sequence set; The foreground image is obtained by segmenting the target image based on the target background; The foreground image is judged as foreground. If the foreground image is judged as foreground within a preset number of times, the foreground image is marked as a false background; otherwise, the foreground image is not marked as a false background. The preset number of times is the number of times the target recognition model processes the images in the image sequence set in sequence. The similarity between the pseudo background and the standard background is calculated. If the similarity is greater than the similarity threshold, the pseudo background is updated as the standard background; otherwise, the standard background is not updated. Images containing pseudo-background regions are extracted from the image sequence set as negative sample datasets, and images containing standard background regions are extracted as positive sample datasets. The negative sample dataset and the positive sample dataset are then combined to obtain the incremental training set. The incremental training set is input into the target recognition model for training, and the parameters of the target recognition model are fine-tuned to obtain a target recognition model that is updated in stages.

5. The deep learning-based dynamic target tracking method according to claim 4, characterized in that, The process of acquiring the target background and standard background of the image sequence set includes: Randomly select pixels from the images in the image sequence set to obtain the initial background; Calculate the expected value and standard deviation of pixels in the image sequence set, and obtain the pixel threshold range based on the expected value and standard deviation; The initial background pixels are anomaly detected. If a pixel in the initial background is within a pixel threshold range, it is determined to be a normal pixel; otherwise, it is determined to be an abnormal pixel, and the abnormal pixel is replaced to obtain the target background. The target background is updated in eight neighborhoods when the number of times the target recognition model processes the images in the image sequence set meets a preset number of times. Based on the standard deviation of pixels, pixel selection is performed on images in the image sequence set to obtain a standard background.

6. The deep learning-based dynamic target tracking method according to claim 1, characterized in that, The improved C2f module is obtained by replacing the Conv module of the C2f module in the YOLOv8 model with a dynamic snake convolution module.

7. The deep learning-based dynamic target tracking method according to claim 1, characterized in that, The improved neck network includes: a first convolutional module, a second convolutional module, and a third convolutional module that are respectively connected to the fourth layer C2f module, the sixth layer C2f module, and the ninth layer SPPF module of the target recognition model, as well as an upsampling module and a downsampling module connected in sequence; The upsampling module includes a first upsampling unit, a second upsampling unit, and a third upsampling unit connected in sequence. Each of the first, second, and third upsampling units includes a Concat module and a C2f module connected in sequence. Specifically, the Concat module of the first upsampling unit concatenates the outputs of the second and third convolutional modules; the Concat module of the second upsampling unit concatenates the outputs of the sixth-layer C2f module, the first convolutional module, and the first upsampling unit; and the Concat module of the third upsampling unit concatenates the outputs of the fourth-layer C2f module and the second upsampling unit. The downsampling module includes a Conv module and a first downsampling unit and a second downsampling unit connected in sequence. Both the first downsampling unit and the second downsampling unit include a Conv module, a Concat module, and a C2f layer connected in sequence. The Concat module in the first downsampling unit is used to concatenate the output results of the Conv module in the first downsampling unit and the C2f module in the second upsampling unit. The Concat module in the second downsampling unit is used to concatenate the output result of the Conv module in the second downsampling unit and the output result of the C2f module in the second upsampling unit that has been processed by the Conv module of the downsampling module.

8. The deep learning-based dynamic target tracking method according to claim 6, characterized in that, The improved neck network operates as follows: The outputs of the ninth-layer SPPF module, the sixth-layer C2f module, and the fourth-layer C2f module in the backbone network are used as inputs to the improved neck structure; the output of the ninth-layer SPPF module is used as input three, the output of the sixth-layer C2f module is used as input two, and the output of the sixth-layer C2f module is used as input one. Input 1, Input 2, and Input 3 are respectively input into the first convolution module, the second convolution module, and the third convolution module for convolution operations to obtain the first feature map, the second feature map, and the third feature map; the second feature map and the third feature map are input into the Concat module of the first upsampling unit for concatenation to obtain the fourth feature map; the fourth feature map is input into the C2f module of the first upsampling unit to obtain the fifth feature map; Input 2, the first feature map, and the fifth feature map are input into the Concat module of the second upsampling unit for concatenation to obtain the sixth feature map; the sixth feature map is input into the C2f module of the second upsampling unit to obtain the seventh feature map; the seventh feature map is concatenated with the first input into the Concat module of the third upsampling unit to obtain the eighth feature map; the eighth feature map is input into the C2f module of the third upsampling unit to obtain the ninth feature map; the ninth feature map is used as the input of the third detection head; The ninth feature map is input into the Conv module of the first downsampling unit to obtain the tenth feature map; the tenth feature map and the seventh feature map are input into the Concat module of the first downsampling unit and concatenated to obtain the eleventh feature map; the eleventh feature map is input into the C2f module of the first downsampling unit to obtain the twelfth feature map; the twelfth feature map is used as the input of the second detection head; The twelfth feature map is input into the Conv module of the first downsampling unit to obtain the thirteenth feature map; the seventh feature map is input into the Conv module of the downsampling module to obtain the fourteenth feature map; the thirteenth and fourteenth feature maps are input into the Concat module of the second downsampling unit and concatenated to obtain the fifteenth feature map; the fifteenth feature map is input into the C2f module of the second downsampling unit to obtain the sixteenth feature map, and the sixteenth feature map is used as the input of the first detection head.

9. A dynamic target tracking device based on deep learning, characterized in that, include: The data acquisition module is used to acquire a set of image sequences of target objects in the target area; The pre-update module is used to update the preset recognition model based on the image sequence set of the target object to obtain the target recognition model; The feature recognition module is used to input the image sequence set into the target recognition model for feature extraction, thereby obtaining the target features corresponding to each image in the image sequence set. The target recognition model is an improvement on the YOLOv8 model, which includes a backbone network, a neck network, and a head network connected in sequence. The C2f module in the eighth layer of the backbone network is replaced with an improved C2f module, and the neck network is replaced with an improved neck network to obtain the target recognition model. The trajectory determination module is used to determine the target trajectory of the target object based on the target features corresponding to each image in the image sequence set.

10. A computer-readable storage medium storing a computer program that, when executed, implements the deep learning-based dynamic target tracking device according to any one of claims 1-8.

Images