A power inspection key target identification and tracking method

Abstract

The application relates to a power inspection key target identification and tracking method, which comprises the following steps: S1: collecting unmanned aerial vehicle inspection images of a power distribution line, using SimCLR for self-supervised feature learning, forming a basic feature platform, using a semi-supervised learning method, generating pseudo labels and then performing manual fine tuning to form a sample library of a distribution network scene S2: pre-processing the labeled image data, including image enhancement and denoising; S3: constructing a target identification model based on a Swin Transformer, training based on the pre-processed image data, introducing multi-task learning, and optimizing the target detection and mask branch to be performed simultaneously; S4: loading the trained target detection and identification model on the unmanned aerial vehicle, detecting the target, and tracking the key target detected and identified based on a target tracking model. The application effectively improves the processing capacity for complex target identification and tracking tasks in power inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image detection, in particular to a power inspection key target recognition and tracking method. BACKGROUND

[0002] In the maintenance and management of modern power systems, efficient and accurate inspection is crucial to ensure the safe operation of the power grid. Traditional manual inspection is time-consuming and labor-intensive, especially when it comes to widely distributed and complex terrain power facilities, its limitations are even more pronounced. Therefore, using unmanned aerial vehicles for power inspection has become a new technical means, which not only can cover areas difficult to reach, but also can reduce the investment of human resources while improving the frequency and accuracy of data collection.

[0003] In the unmanned aerial vehicle power inspection task, target recognition and tracking technology plays a crucial role. The core task of this technology is to automatically identify power equipment (such as power poles, transformers, insulators, etc.) from video or image data collected by unmanned aerial vehicles and continuously track during the inspection process, but existing target recognition and tracking technology still faces many challenges, such as high real-time requirements, variable environments (such as weather, terrain), and diversity of power grid equipment. SUMMARY

[0004] In order to solve the above problems, the purpose of the present application is to provide a power inspection key target recognition and tracking method, which effectively improves the processing capability of complex target recognition and tracking tasks in power inspection.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] A power inspection key target recognition and tracking method, comprising the following steps:

[0007] S1: Collecting unmanned aerial vehicle inspection images of distribution lines, sorting the image data into a distribution line unmanned aerial vehicle inspection image library according to the scene, and using SimCLR for self-supervised feature learning to form a basic feature platform, using a semi-supervised learning method to generate pseudo-labels and then performing manual fine tuning to form a sample library of distribution network scenes;

[0008] S2: Preprocessing the labeled image data, including image enhancement and denoising; adjusting the contrast, brightness and using adaptive histogram equalization for image enhancement, and using non-local mean for denoising;

[0009] S3: Constructing a target recognition model based on Swin Transformer, training based on preprocessed image data, introducing multi-task learning, and optimizing target detection and mask branch at the same time;

[0010] The target recognition model comprises a backbone network Swin Transformer and two sequence branches, a first sequence branch for target detection and a second sequence branch for target instance segmentation;

[0011] Features are extracted by the Swin Transformer, which adopts a hierarchical network structure to perform multi-scale feature extraction in layers; a moving window mechanism is used to perform local self-attention calculation, reducing the calculation cost and capturing global features;

[0012] Features are extracted by sharing the backbone network, and after the features are extracted by the backbone Swin Transformer, the features are sent to two branches, a target detection branch that proposes candidate regions by a region proposal network RPN and a target instance segmentation branch based on instance segmentation of Mask R-CNN; and multi-task loss optimization is performed;

[0013] According to the preprocessed image data, training is performed using a mixed precision and distributed strategy to accelerate;

[0014] S4: The trained target detection recognition model is loaded onto the unmanned aerial vehicle to detect targets, and the key targets detected by the recognition are tracked based on a target tracking model.

[0015] Further, S1 specifically comprises:

[0016] A large number of images of power distribution lines are collected by the unmanned aerial vehicle, covering different scenes, weather conditions and angles;

[0017] The image data is sorted by scene to establish a structured image library;

[0018] SimCLR is used for self-supervised feature learning to obtain features of samples and construct a basic feature platform H for semi-supervised learning;

[0019] Based on the basic feature platform H, a semi-supervised learning method Label Propagation algorithm is used to perform label propagation, and high-confidence pseudo-label data is screened out using confidence;

[0020] The basic feature platform H is used to generate initial labeling to generate part of the labeled data Ylabeled;

[0021] Unlabeled data Yunlabeled is used in combination with the basic feature platform H

[0022] A similarity matrix S is constructed to describe the similarity between data,

[0023] S ij =exp(-γ||h i -h j|| 2 );

[0024] where γ is a tuning parameter of the similarity measure; h i , h j is the feature representation;

[0025] Label propagation formula:

[0026] Y (t+1) = αSY (t) + (1-α)Y (0) ;

[0027] where Y (0) is the initial label matrix; Y (t) is the current label matrix; α is a hyperparameter that adjusts the strength of label propagation;

[0028] According to the confidence, reliable pseudo labels are screened, and the final high-quality sample library is formed by manual correction or fine-tuning.

[0029] Further, SimCLR is used for self-supervised feature learning to obtain the features of the samples and construct a basic feature platform H for semi-supervised learning, as follows:

[0030] For each image x, an augmented transformation is performed, including rotation, cropping and color disturbance, to generate two views x i and x j ;

[0031] Feature representation is extracted by neural network f θ :

[0032] h i = f θ (x i ), h j = f θ (x j );

[0033] where θ represents the parameters of the neural network;

[0034] The feature is mapped to a vector space using a projection head g φ :

[0035] z i = g φ (h i ), z j = g φ (h j );

[0036] where φ represents the parameters of the projection head;

[0037] The model is optimized by contrastive learning loss:

[0038]

[0039] wherein z i ,z j is the feature vector obtained after projection; tau is a temperature parameter;

[0040] Obtain the feature output of all samples, and construct a feature platform H for semi-supervised learning.

[0041] Further, S2 is specifically:

[0042] Obtain the labeled image data, adjust the contrast of the image through linear transformation, let the original image be I, and the enhanced image be I', and the contrast adjustment is expressed as:

[0043] I' = a I + b;

[0044] Wherein a is the gain coefficient for adjusting the contrast, and b is the brightness offset;

[0045] Apply CLAHE to the image for local contrast enhancement;

[0046] Use NLM algorithm to remove image noise, for each pixel, use the whole image to denoise it, the formula is as follows:

[0047]

[0048] wherein, is the pixel value after denoising; I'(y) is the pixel value of the input image at position y, Omega is the entire image domain; w(x, y) is the weight, defined as:

[0049]

[0050] wherein N x and N y are the adjacent windows of points x and y respectively, Z(x) is a normalization constant, h is a decay coefficient, which controls the smoothness; represents the sum of the square differences of the intensity of the paired pixels in the windows N x and N y .

[0051] Further, apply CLAHE to the image for local contrast enhancement, specifically as follows:

[0052] Divide the image into several non-overlapping grids;

[0053] Calculate the local histogram for each grid; set the clipping limit clip limit, clip the level whose frequency exceeds the limit in the histogram, and the calculation formula is:

[0054] H clipped (i) = min(H(i), clip limit);

[0055] where H(i) is the frequency of the i-th histogram gray level;

[0056] The excess is evenly distributed to the entire histogram to avoid noise amplification;

[0057] The equalized cumulative distribution function CDF is calculated using the clipped histogram, and the original pixel value is mapped to the equalized new value through CDF:

[0058]

[0059] where L is the maximum value of the gray level (usually 255), CDF min and CDF max are the minimum and maximum values of CDF, respectively;

[0060] For each pixel, the final equalization result is obtained from the adjacent grid by bilinear interpolation to avoid image block effect, and the final image is composed of all interpolated grids.

[0061] Further, the target detection branch proposes candidate regions through the region proposal network RPN, and the target instance segmentation branch is based on the instance segmentation of Mask R-CNN; and multi-task loss optimization is performed, as follows:

[0062] The target detection branch includes a region proposal part and a detection part, and the region proposal network RPN generates candidate boxes for further detection, generates anchor boxes on the feature map through sliding network, and outputs suitable target regions;

[0063] For the input feature F, a 3x3 convolution is used to generate an intermediate feature;

[0064] A 1x1 convolution is used to give 2k classification scores and 4k bounding box regression coordinates correction, k being the number of anchor boxes at each position;

[0065] Irregular proposal regions are uniformly pooled into feature maps of the same size, and a fully connected network is used to classify the target class and correct the bounding box prediction for each RoI, and the loss function is:

[0066]

[0067] where p is the target classification probability, y is the label, t* and t are the real and predicted bounding box parameters, respectively; L cls is the classification loss function; L reg is the regression loss function;

[0068] Mask R-CNN uses a convolutional network to predict a mask on the RoI, after the original RoI pooling, a convolutional network is added to generate a pixel-level target mask, and the mask loss is:

[0069]

[0070] Where M ij is the true mask pixel value, 0 represents the background, and 1 represents the foreground; is the predicted mask pixel probability; m x m is the resolution of the mask.

[0071] The comprehensive multi-task loss is:

[0072]

[0073] Where λ cls , λ reg , λ mask are the weight coefficients corresponding to the loss.

[0074] Further, S4 is specifically:

[0075] According to the trained target recognition model, each frame input by the camera is processed to identify and obtain the position information and confidence of each target;

[0076] Deep SORT method is used for continuous tracking;

[0077] The PID control algorithm is used to adjust the flight path of the unmanned aerial vehicle, so that the unmanned aerial vehicle can continuously follow the target.

[0078] Further, the Deep SORT method is used for continuous tracking, specifically as follows:

[0079] The detection results of the current frame are matched with the previous trajectories, and the Hungarian algorithm is used for optimal allocation:

[0080] a. Use IoU to measure the overlap between each detection result and the existing trajectory, and the element C(i,j) of the cost matrix is the matching cost, which is 1-IoU;

[0081] b. Use the Hungarian algorithm on the cost matrix to find the matching with the smallest total cost;

[0082] c. Determine which detection results match which existing trajectories through the output of the Hungarian algorithm;

[0083] d. The detection results that are not matched may be initialized as new trajectories, and the trajectories that are not matched may be marked as lost;

[0084] Initialize a Kalman filter for each tracked target to maintain state prediction and update;

[0085] The visual features of the target are acquired using the target detection branch, and the target track is updated;

[0086] For unmatched detection results, a new track and Kalman filter are initialized, the existing track that is not matched is continued, and the track that is lost for a long time is terminated after a set number of frames.

[0087] The present application has the following beneficial effects:

[0088] 1. The present application effectively improves the processing capability for complex target recognition and tracking tasks in power inspection;

[0089] 2. The present application combines CLAHE and NLM algorithms for preprocessing, improves image quality, makes the image more robust in machine learning tasks, and improves the understanding ability of the model for complex scenes, which can effectively process image data in power distribution line inspection, and provide clearer and more information-rich input for subsequent recognition models;

[0090] 3. The target recognition model constructed based on Swin Transformer effectively provides excellent performance in complex target recognition tasks, and through the combination of Hungarian algorithm matching, Kalman filter prediction update and track management strategy, the unmanned aerial vehicle can perform stable target tracking in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0091] Figure 1 The present application is a method flowchart;

[0092] Figure 2 The present application is a tracking schematic diagram in an embodiment. DETAILED DESCRIPTION

[0093] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0094] A power inspection key target recognition and tracking method, comprising the following steps:

[0095] S1: Collecting unmanned aerial vehicle inspection images of power distribution lines, sorting the image data into power distribution line unmanned aerial vehicle inspection image library according to the scene, using SimCLR for self-supervised feature learning to form a basic feature platform, using a semi-supervised learning method to generate pseudo labels and then performing manual fine tuning to form a sample library of distribution network scenes;

[0096] S2: Preprocessing the labeled image data, including image enhancement and denoising; adjusting the contrast, brightness and using adaptive histogram equalization for image enhancement, and using non-local mean for denoising;

[0097] S3: Constructing a target recognition model based on Swin Transformer, training based on pre-processed image data, introducing multi-task learning, optimizing target detection and mask branch at the same time;

[0098] The target recognition model includes a backbone network Swin Transformer and two sequence branches, sequence branch one for target detection, and branch two for target instance segmentation;

[0099] Extract features through Swin Transformer, which uses a hierarchical network structure to extract multi-scale features in layers; use the moving window mechanism to perform self-attention calculation locally, reduce the calculation cost and capture global features;

[0100] Extract features through a shared backbone network, after the backbone Swin Transformer extracts features, these features are sent to two branches, the target detection branch proposes candidate regions through the region proposal network RPN, and the target instance segmentation branch is based on the instance segmentation of Mask R-CNN; and multi-task loss optimization is performed;

[0101] According to the pre-processed image data, training is carried out, and mixed precision and distributed strategy are used to accelerate;

[0102] S4: Mount the trained target detection and recognition model on the unmanned aerial vehicle to detect the target, and track the key target detected by the target tracking model.

[0103] In this embodiment, S1 is specifically:

[0104] Use the unmanned aerial vehicle to collect a large number of images of power distribution lines, covering different scenes, weather conditions and angles;

[0105] Organize the image data by scene (such as city, countryside and wild) and establish a structured image library;

[0106] Use SimCLR for self-supervised feature learning to obtain the features of the samples and build a basic feature platform H for semi-supervised learning;

[0107] Based on the basic feature platform H, use the semi-supervised learning method Label Propagation algorithm to perform label propagation, and use the confidence to screen out high-confidence pseudo-label data;

[0108] Use the basic feature platform H to generate part of the labeled data Ylabeled;

[0109] Use unlabeled data Yunlabeled and combine the basic feature platform H

[0110] A similarity matrix S is constructed to describe the similarity between data,

[0111] S ij = exp(-γ||h i -h j || 2 );

[0112] where γ is a tuning parameter of the similarity measure; h i , h j are feature representations;

[0113] Label propagation formula:

[0114] Y (t+1) = αSY (t) +(1-α)Y (0) ;

[0115] where Y (0) is the initial label matrix; Y (t) is the label matrix at the current time; α is a hyperparameter that adjusts the strength of label propagation;

[0116] According to the confidence, reliable pseudo labels are screened, and the final high-quality sample library is formed by manual correction or fine-tuning.

[0117] In this embodiment, SimCLR is used for self-supervised feature learning to obtain the features of the samples and construct a basic feature platform H for semi-supervised learning, as follows:

[0118] For each image x, an augmented transformation is performed, including rotation, cropping and color perturbation, to generate two views x i and x j ;

[0119] Feature representations are extracted by a neural network f θ :

[0120] h i = f θ (x i ), h j = f θ (x j );

[0121] where θ represents the parameters of the neural network; in this embodiment, a convolutional neural network (CNN) is used;

[0122] A projection head g φ is used to map the features to a vector space:

[0123] z i = g φ (hi ), z j = g φ (h j );

[0124] wherein φ represents the parameters of the projection head;

[0125] Optimize the model by contrast learning loss:

[0126]

[0127] wherein z i , z j are the feature vectors obtained after projection; τ is a temperature parameter;

[0128] Obtain the feature output of all samples, and construct a feature platform H for semi-supervised learning.

[0129] In the embodiment, S2 specifically is:

[0130] Obtain the labeled image data, adjust the contrast of the image through linear transformation, let the original image be I, and the enhanced image be I', and the contrast adjustment is expressed as:

[0131] I'=a*I+b;

[0132] Wherein a is a gain coefficient for adjusting the contrast, and b is a brightness offset;

[0133] Apply CLAHE to the image for local contrast enhancement;

[0134] Use the NLM algorithm to remove image noise to reduce the noise influence brought by the enhancement process, for each pixel, use the whole image to denoise it, and the formula is as follows:

[0135]

[0136] wherein, is the pixel value after denoising; I'(y) is the pixel value of the input image at position y, Ω is the entire image domain; w(x,y) is the weight, defined as:

[0137]

[0138] wherein N x and N y are the adjacent windows of points x and y, Z(x) is a normalization constant, h is a decay coefficient, and controls the smoothness; represents the sum of the square differences of the intensities of the paired pixels in the windows N x and N y .

[0139] In this embodiment, CLAHE is applied to the image for local contrast enhancement, specifically as follows:

[0140] The image is divided into several non-overlapping grids;

[0141] The local histogram of each grid is calculated; the clip limit is set, and the levels of the frequency exceeding the limit in the histogram are clipped, and the calculation formula is:

[0142] H clipped (i) = min(H(i), clip limit);

[0143] Where H(i) is the gray level frequency of the i-th histogram;

[0144] The excess is evenly distributed to the entire histogram to avoid noise amplification;

[0145] The equalized cumulative distribution function CDF is calculated using the clipped histogram, and the original pixel value is mapped to the equalized new value through CDF:

[0146]

[0147] Where L is the maximum value of the gray level (usually 255), CDF min And CDF max are the minimum and maximum values of CDF, respectively;

[0148] For each pixel, the final equalization result is obtained from the adjacent grid by bilinear interpolation to avoid image block effect, and the final image is composed of all interpolated grids.

[0149] In this embodiment, the target detection branch proposes candidate regions through the region proposal network RPN, and the target instance segmentation branch is based on the instance segmentation of Mask R-CNN; and multi-task loss optimization is performed, specifically as follows:

[0150] The target detection branch includes a region proposal part and a detection part, and the region proposal network RPN generates candidate boxes for further detection, generates anchor boxes on the feature map through sliding network, and outputs suitable target regions;

[0151] For the input feature F, a 3x3 convolution (shared convolution) is used to generate an intermediate feature;

[0152] A 1x1 convolution is used to give 2k classification scores and 4k bounding box regression coordinates correction, and k is the number of anchor boxes at each position;

[0153] The irregular proposal regions are uniformly pooled into feature maps of the same size, and a fully connected network is used to classify the target categories and correct the bounding box prediction for each RoI, and the loss function is:

[0154]

[0155] where p is the target classification probability, y is the label, t* and t are the real and predicted bounding box parameters respectively; L cls is the classification loss function; L reg is the regression loss function;

[0156] Mask R-CNN uses a convolutional network to predict a mask map on the RoI, after the original RoI pooling, a convolutional network is added to generate a pixel-level target mask, and the mask loss is:

[0157]

[0158] where M ij is the real mask pixel value, 0 represents the background, and 1 represents the foreground; is the predicted mask pixel probability; m x m is the resolution of the mask.

[0159] The comprehensive multi-task loss is:

[0160]

[0161] where λ cls , λ reg , λ mask are the weight coefficients corresponding to the loss.

[0162] In this embodiment, S4 is specifically:

[0163] According to the trained target recognition model, each frame input by the camera is processed, and the position information and confidence of each target are recognized and obtained;

[0164] The Deep SORT method is used for continuous tracking;

[0165] The PID control algorithm is used to adjust the flight path of the unmanned aerial vehicle, so that the unmanned aerial vehicle can continuously follow the target.

[0166] In this embodiment, the Deep SORT method is used for continuous tracking, specifically as follows:

[0167] The detection result of the current frame is matched with the previous trajectory, and the Hungarian algorithm is used for optimal allocation:

[0168] a. The overlap between each detection result and the existing trajectory is measured using IoU (Intersection over Union), and the element C(i,j) of the cost matrix is the matching cost, which is 1-IoU;

[0169] b. Find the matching with the minimum total cost on the cost matrix using the Hungarian algorithm;

[0170] c. Determine which detections match which existing tracks through the output of the Hungarian algorithm;

[0171] d. Detections that do not match can be initialized as new tracks, and tracks that are not matched can be marked as lost;

[0172] Initialize a Kalman filter for each tracked object to maintain state prediction and update;

[0173] Use the object detection branch to obtain visual features of the object, and update to the object track;

[0174] For unmatched detections, initialize new tracks and Kalman filters, continue existing tracks that are not matched, and destroy them after a set number of frames, and terminate long-lost tracks.

[0175] Those skilled in the art will appreciate that embodiments of the application can be devised for use with various computer interpretable media. Accordingly, one embodiment provides a computer program product which can include a computer readable storage medium having stored thereon instructions that can be executed by a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which when executed by the computer or other programmable data processing apparatus, implement the functions / acts specified in the flowcharts and / or block diagrams.

[0176] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowcharts and / or block diagrams. Figure 1 The flowcharts and block diagrams in the various figures can implement or support sub-processes, methods or algorithms implemented by computer program instructions. However, the computer program instructions can not necessarily be directly Figure 1 The flowcharts and block diagrams in the various figures can implement or support sub-processes, methods or algorithms implemented by computer program instructions. However, the computer program instructions can not necessarily be directly

[0177] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowcharts and / or block diagrams. Figure 1 The flowcharts and block diagrams in the various figures can implement or support sub-processes, methods or algorithms implemented by computer program instructions. However, the computer program instructions can not necessarily be directly Figure 1 The flowcharts and block diagrams in the various figures can implement or support sub-processes, methods or algorithms implemented by computer program instructions. However, the computer program instructions can not necessarily be directly

[0178] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the block Figure 1 one flow or multiple flows and / or the functions specified in the block

[0179] The above description is only the preferred embodiment of the present application, not other forms of the present application, any skilled in the art can use the above disclosed technical content to change or modify as equivalent embodiments of equivalent changes. But any simple modification, equivalent change and modification of the above embodiments without departing from the technical solution of the present application, according to the technical essence of the present application, still belongs to the protection scope of the technical solution of the present application.

Claims

1. A method for power inspection key target identification and tracking, characterized in that, The method comprises the following steps: S1: Collecting unmanned aerial vehicle inspection images of power distribution lines, sorting the image data into power distribution line unmanned aerial vehicle inspection image library according to scenes, and using SimCLR for self-supervised feature learning to form a basic feature platform; using a semi-supervised learning method to generate pseudo-labels and then performing manual fine tuning to form a sample library of power distribution scenes; S2: Preprocessing the labeled image data, including image enhancement and denoising; adjusting the contrast, brightness and using adaptive histogram equalization for image enhancement, and using non-local mean for denoising; S3: Constructing a target recognition model based on Swin Transformer, training based on the preprocessed image data, introducing multi-task learning, and optimizing target detection and mask branch at the same time; The target recognition model comprises a backbone network Swin Transformer and two sequence branches, and the sequence branch one is used for target detection, and the sequence branch two is used for target instance segmentation; The Swin Transformer extracts features, adopts a hierarchical network structure, performs multi-scale feature extraction in layers, uses a moving window mechanism to perform local self-attention calculation, reduces the calculation cost and captures global features; The features are extracted by the shared backbone network, and after the features are extracted by the backbone Swin Transformer, the features are sent to the two branches, the target detection branch proposes candidate regions through the region proposal network RPN, and the target instance segmentation branch is based on the instance segmentation of Mask R-CNN; and multi-task loss optimization is performed; According to the preprocessed image data, training is performed, and mixed precision and distributed strategy are adopted to accelerate; S4: The trained target recognition model is carried on the unmanned aerial vehicle to detect the target, and the key target detected and recognized is tracked based on a target tracking model.

2. The method of claim 1, wherein, The S1 is specifically: A large number of images of power distribution lines are collected by using an unmanned aerial vehicle, covering different scenes, weather conditions and angles; The image data is sorted according to scenes to establish a structured image library; SimCLR is used for self-supervised feature learning to obtain the features of samples and construct a basic feature platform H for semi-supervised learning; Based on the basic feature platform H, a label propagation algorithm is used to perform label propagation, and high-confidence pseudo-label data is screened out using confidence; Initial labeling is performed using the basic feature platform H to generate part of the labeled data Ylabeled; Unlabeled data Yunlabeled is used in combination with the basic feature platform H A similarity matrix S is constructed to describe the similarity between data, ； wherein, is a tuning parameter of the similarity measure; , is a feature representation; The label propagation formula is: ； where Y (0) is the initial label matrix; Y (t) is the label matrix at the current time; and a is a hyperparameter for adjusting the label propagation intensity. Reliable pseudo-labels are screened out according to the confidence, and are manually corrected or fine-tuned to form a final high-quality sample library.

3. The method of claim 2, wherein, The SimCLR is used for self-supervised feature learning to obtain the features of samples and construct a basic feature platform H for semi-supervised learning, and the specific process is as follows: For each image x Perform augmentation transformations, including rotation, cropping, and color perturbation, to produce two views. x i and x j ; By neural network f θ Extracting feature representations: ； Where θ represents the parameters of the neural network; Using a projection head Mapping features to vector space: ； wherein parameters representative of the projection head; The model is optimized by contrast learning loss: ； where z i ,z j is the feature vector obtained after projection; τ is a temperature parameter; Obtain the feature output of all samples, and construct a feature platform H for semi-supervised learning.

4. The method of claim 1, wherein, The S2 is specifically: Obtain the labeled image data, adjust the image contrast through linear transformation, let the original image be I, and the enhanced image be I', and the contrast adjustment is expressed as: ； Wherein, a is the gain coefficient for adjusting the contrast, and b is the brightness offset; Apply CLAHE to the image for local contrast enhancement; Use NLM algorithm to remove image noise, for each pixel, use the whole image to denoise, the formula is as follows: ； wherein is the de-noised pixel value; is the pixel value of the input image at position y, Ω is the entire image domain; w(x, y) is the weight, defined as: ； where N x and N y are the adjacent windows of points x and y, respectively, Z(x) is a normalization constant, h is a decay coefficient that controls the smoothness, and represents the sum of squared differences of pixel intensities of pairs of windows N x and N y .

5. The method of claim 4, wherein, The local contrast enhancement of the image by CLAHE is specifically as follows: Divide the image into several non-overlapping grids; Calculate the local histogram for each grid; Set the clipping limit clip limit, clip the frequency of the level in the histogram that exceeds this limit, and the calculation formula is: H clipped (i)=min(H(i),clip limit) ； Wherein, H(i) is the gray level frequency of the i-th histogram; Distribute the excess part evenly to the whole histogram to avoid noise amplification; Calculate the equalized cumulative distribution function CDF using the clipped histogram, and map the original pixel value to the equalized new value through CDF: ； where L is the maximum value of the gray scale, CDF min and CDF max are the minimum and maximum values of the CDF, respectively; For each pixel, get the final equalization result from the adjacent grid through bilinear interpolation to avoid image block effect, and the final image is composed of all interpolated grids.

6. The method of claim 1, wherein, The target detection branch proposes candidate regions through the region proposal network RPN, and the target instance segmentation branch is based on the instance segmentation of Mask R-CNN; and multi-task loss optimization is performed, specifically as follows: The target detection branch includes a region proposal part and a detection part, the region proposal network RPN generates candidate boxes for further detection, generates anchor boxes on the feature map through sliding network, and outputs suitable target regions; Use 3x3 convolution to generate intermediate features on the input features F; Use 1x1 convolution to give 2k classification scores and 4k bounding box regression coordinates correction, k is the number of anchor boxes at each position; Uniformly pool the irregular proposal regions into feature maps of the same size, use a fully connected network to classify the target categories and correct the bounding box prediction for each RoI, and the loss function is: ； Wherein, p is the target classification probability, y is the label, t* and t are the real and predicted bounding box parameters respectively; L cls is a classification loss function; L reg is a regression loss function; Mask R-CNN uses a convolutional network to predict a mask map on the RoI, after the original RoI pooling, a convolutional network is added to generate a pixel-level target mask, and the mask loss is: ； where M ij is the true mask pixel value, 0 indicates background and 1 indicates foreground; is the predicted mask pixel probability; m x m is the resolution of the mask; Integrate the multi-task loss: ； wherein, λ cls ,λ reg , λ mask is the weight coefficient corresponding to the loss.

7. The method of claim 6, wherein, The S4 is specifically: According to the trained target recognition model, process each frame input by the camera, recognize and obtain the position information and confidence of each target; Use the Deep SORT method for continuous tracking; Use the PID control algorithm to adjust the flight path of the unmanned aerial vehicle, so that the unmanned aerial vehicle can continuously follow the target.

8. The method of claim 7, wherein, The Deep SORT method is used for continuous tracking, specifically as follows: Match the detection results of the current frame with the previous trajectory, and use the Hungarian algorithm for optimal allocation: a. The overlap between each detection and existing tracks is measured using IoU, the element of cost matrix C(i,j) is the matching cost, which is 1 - IoU; b. The Hungarian algorithm is used on the cost matrix to find the matching with the minimum total cost; c. The output of the Hungarian algorithm determines which detections match which existing tracks; d. The detections that are not matched can be initialized as new tracks, and the tracks that are not matched can be marked as lost; A Kalman filter is initialized for each tracked target to maintain state prediction and update; The visual features of the target are obtained using the target detection branch, and updated to the target track; For unmatched detections, new tracks and Kalman filters are initialized, and existing tracks that are not matched are continued, and are destroyed after a set number of frames, and long-lost tracks are terminated.

Images