Unmanned aerial vehicle industrial inspection method based on transfer learning, product, medium and equipment

Abstract

The invention discloses an unmanned aerial vehicle industrial inspection method based on transfer learning, a product, a medium and equipment, and relates to the technical field of industrial inspection, and the method comprises the steps: obtaining an unmanned aerial vehicle inspection data set, and carrying out the preprocessing; pre-training a label classifier based on the pre-processed training set to obtain a trained label classifier; a trained label classifier is adopted to classify the preprocessed test set, and a classified test set is obtained; inputting the preprocessed training set and the classified test set into a transfer learning model to perform mixed feature antagonism learning training to obtain a trained transfer learning model; and through the trained transfer learning model, analyzing the data collected in the actual electric power inspection scene, and completing the industrial inspection of the unmanned aerial vehicle. According to the invention, the efficiency and accuracy of unmanned aerial vehicle industrial inspection can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of industrial inspection technology, and in particular to a method, product, medium and equipment for industrial inspection using drones based on transfer learning. Background Art

[0002] Industrial inspection is an important part of ensuring the safe and efficient operation of industrial facilities. Traditional industrial inspections mostly rely on manual inspections, but with the development of technology, unmanned aerial vehicles (UAVs) have long been widely used in the field of industrial inspections, especially in difficult-to-access or high-risk environments.

[0003] Despite the increasing use of drones in industrial inspections, existing drone inspection systems still face multiple technical challenges. Since drones need to fly stably in various complex industrial environments, this places high demands on their navigation and control systems. When drones conduct industrial inspections, the image data captured in different environments (such as different lighting conditions, different backgrounds, etc.) and different drone equipment (such as different camera parameters) may be visually very different. These differences can cause the performance of a model trained in one environment to drop sharply when applied to another environment with different appearance or equipment characteristics. In addition, inspection tasks usually require the identification and analysis of various complex industrial equipment states, which requires not only high-precision image acquisition, but also powerful image processing and data analysis capabilities. Moreover, in actual industrial inspection scenarios, it is very difficult and costly to obtain a large amount of labeled data, especially for each specific application environment. Therefore, although drones can improve the safety of inspections, reduce labor costs, and increase the frequency of inspections, the efficiency and accuracy of drone inspections are still limited by the sensor technology and data processing capabilities of their payloads.

[0004] In summary, due to the diversity of industrial equipment and environments, research on improving the high adaptability and generalization ability of inspection systems is crucial to improve the efficiency and accuracy of UAV inspections. Summary of the invention

[0005] The purpose of this application is to provide a UAV industrial inspection method, product, medium and equipment based on transfer learning, which can effectively improve the efficiency and accuracy of UAV industrial inspection.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for industrial inspection using a drone based on transfer learning, comprising:

[0008] Obtaining a drone inspection data set and preprocessing it; the drone inspection data set includes a training set and a test set;

[0009] Pre-train the label classifier based on the preprocessed training set to obtain a trained label classifier;

[0010] The preprocessed test set is classified using the trained label classifier to obtain a classified test set;

[0011] Inputting the preprocessed training set and the classified test set into the transfer learning model for mixed feature adversarial learning training to obtain a trained transfer learning model; the transfer learning model includes two feature extractors, two trained label classifiers, a gradient reversal layer, and a scale discriminator;

[0012] The trained transfer learning model is used to analyze the data collected from actual power inspection scenarios and complete drone industrial inspections.

[0013] In a second aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the transfer learning-based drone industrial inspection method.

[0014] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the transfer learning-based drone industrial inspection method.

[0015] In a fourth aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the transfer learning-based drone industrial inspection method.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] The present application provides a method, product, medium and equipment for industrial inspection by unmanned aerial vehicles (UAVs) based on transfer learning. The method obtains and preprocesses the UAV inspection data set, adopts domain adaptation technology, and uses the labeled data on the data-rich training set to help effectively learn on the test set that may have almost no labeled data, thereby solving the problem of scarce labeled data; by inputting the preprocessed training set and the classified test set into the transfer learning model for mixed feature adversarial learning training, the transfer learning model learns a common representation between the training set and the test set, reduces the difference between domains, and improves the generalization ability of the transfer learning model; then, the trained transfer learning model is used to analyze the data collected in the actual power inspection scene to complete the UAV industrial inspection, which can effectively improve the efficiency and accuracy of the UAV industrial inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flow chart of the UAV industrial inspection method based on transfer learning provided in this application;

[0020] Figure 2 A schematic diagram of the principle of the UAV industrial inspection method based on transfer learning provided in this application;

[0021] Figure 3 Schematic diagram of the principle of the Self-Attention module provided in this application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0024] As an effective machine learning strategy, transfer learning provides new possibilities for solving problems in background technologies. Transfer learning can use models pre-trained on large data sets, fine-tune them with a small amount of target domain data, and quickly adapt to new tasks. This can not only significantly improve the image recognition accuracy of drones, but also speed up their learning and reduce their dependence on large amounts of labeled data. For example, a visual recognition model trained in a general scenario can be quickly converted into a model dedicated to detecting a specific type of industrial equipment through transfer learning. Researching and implementing efficient transfer learning methods is crucial to improving the performance of drone industrial inspections. This will directly affect the accuracy, efficiency, and economy of inspections, and thus affect the safety and stability of the entire industrial production. An effective transfer learning strategy can enable drones to maintain high performance in different industrial environments, especially when facing equipment or complex environments that have never been seen before.

[0025] like Figure 1As shown, the present application discloses a UAV industrial inspection method based on transfer learning, including:

[0026] Step 1: Obtain the drone inspection dataset and preprocess it.

[0027] The drone inspection dataset obtained in this application is divided into a training set (Training set) and a test set (TestingSet). The Training set is a large-scale visual dataset (such as ImageNet) that collects images of related wire insulator cracks, line flashovers, etc. The test set can be photos of power line faults taken by drones in actual sites, and labels need to be set for the training set.

[0028] Specifically, for the data acquisition of the training set, the following types of images should be collected from large-scale visual datasets and precise labels should be applied. The images mainly include images of cracks in wire insulators, images of line flashovers, and images of normal power lines. In addition, for the image type, images of power facilities in different weather conditions (especially in bad weather) need to be specially acquired in large-scale visual datasets to enhance the robustness of the transfer learning model. At the same time, it is also necessary to consider images of special situations under occlusion (such as tree branches, buildings, etc.) or partially occluded power facility images to train the transfer learning model to handle complex scenes. The images acquired are not only new ones, but also images of power facilities that are aged, worn, or have obvious signs of damage. After the training set data is acquired, precise labels should be applied to each image, and each image needs to have detailed labels about the problems, such as the location of the cracks, the type of flashover, etc.

[0029] For the data acquisition of the test set, the focus should be on images taken directly from drones in actual power line inspection scenarios. These images will reflect various conditions of power facilities in the real world, including cracks in wire insulators, flashover events of lines, and normal power line conditions. In order to ensure that the test set can fully evaluate the performance of the model under various actual conditions, the images should include shooting results under different weather conditions (especially bad weather), as well as taking into account possible occlusions (such as tree branches or buildings) and aging or damage of facilities.

[0030] After obtaining the images in the drone inspection dataset, all images need to be resized to 64×64, and the number of images in the training set and the test set should be set in a 1:1 ratio.

[0031] Since data images in power inspections face problems such as imbalanced numbers and small amounts of data, it is necessary to preprocess the drone inspection data set and increase the number of images through data enhancement. At the same time, data enhancement can also be used to simulate problems that may occur in actual scenarios under different environments. The details are as follows:

[0032] 1. Add Gaussian noise

[0033] Gaussian noise is added to simulate image quality issues in the real world due to various factors such as environmental noise, device quality, lighting changes, etc. This processing enables the transfer learning model to adapt to noisy input data, thereby improving its robustness in practical applications.

[0034] Specifically, a random value generated by Gaussian distribution is added to each pixel point of the image of the acquired drone inspection dataset. Assuming that the original pixel value is p and the noise value is n (n obeys a Gaussian distribution with a mean of 0 and a standard deviation of σ), the pixel value p′ after noise addition is:

[0035] p′=p+n (1)

[0036] Among them, σ is used to control the noise intensity and is generally adjusted according to actual conditions.

[0037] 2. Image rotation

[0038] Image rotation simulates the situation that drones may shoot power facilities from different angles in actual operations. This not only increases the insensitivity of the transfer learning model to the image orientation, but also expands the viewing angle coverage of the transfer learning model.

[0039] Specifically, by rotating the image of the acquired drone inspection dataset, as shown in formula (2), the presentation angle of the object in the image is changed, which is very useful for identifying the same object observed from various different angles.

[0040]

[0041] where c x and c y is the center coordinate of the image, θ is the rotation angle, and M is the affine transformation matrix used to describe the image rotation, which can rotate the image according to the specified angle θ. The affine transformation matrix here not only takes into account the rotation angle, but also the position of the image rotation center (c x , c y ), use the affine transformation matrix M to transform the coordinates of each pixel in the image.

[0042] As a specific embodiment, after adding Gaussian noise to each image in the training set and the test set, the images are rotated by 30°, 45°, 60°, and 75° respectively, and the acquired drone inspection data set is expanded to 10 times the original. The images after adding Gaussian noise will increase randomness at the pixel level. This preliminary data perturbation helps the transfer learning model ignore tiny, unimportant image details and focus on more macro and important features. When these "noise" images are further rotated, the transfer learning model must recognize and understand the objects under the same noise influence from multiple angles, which greatly increases its adaptability and generalization ability to environmental changes.

[0043] Step 2: Pre-train the label classifier based on the preprocessed training set to obtain a trained label classifier.

[0044] This application uses the preprocessed training set data to preliminarily train the label classifier. This preliminary training process is a standard supervised learning process. The purpose is to enable the label classifier to achieve a higher accuracy rate on the training set data with known labels, and then generate pseudo labels for the test set data, as shown in step 3.

[0045] Step 3: Use the trained label classifier to classify the preprocessed test set to obtain the classified test set.

[0046] Specifically, the test set data is classified by using the label classifier trained in step 2, and the confidence threshold is set to 0.9 to generate a pseudo label. That is, when the probability of the test set being solved by the trained label classifier is greater than 0.9, the generated label is used as a pseudo label.

[0047] Step 4: Input the preprocessed training set and the classified test set into the transfer learning model for mixed feature adversarial learning training to obtain a trained transfer learning model.

[0048] The transfer learning model of this application is mainly composed of a feature extractor, a label classifier (Class Classifier), a gradient reversal layer (GRL) and a ratio discriminator (discriminator). The main process is: the preprocessed training set and test set data are all passed through the feature extractor: the preprocessed training set is input into the first feature extractor, and the classified test set is input into the second feature extractor; the feature extractor is composed of a convolutional neural network (CNN) module and a self-attention module. The convolutional neural network extracts different feature subsets from the original images in the preprocessed training set and test set. The Self-Attention module is used to effectively capture long-distance dependencies and understand complex input structures. The feature extractor extracts useful features from the input data so that these features can be used for subsequent classification tasks. The feature extractor inputs the features extracted from the training set and the test set into the label classifier to predict the correct label (category in the classification task): the training set features are input into the first trained label classifier, and the test set features are input into the second trained label classifier; at the same time, the features extracted from the training set and the test set are mixed in a certain proportion and then input into the proportion discriminator through the gradient reversal layer to determine whether the data comes from the test set or the training set. The core part of the transfer learning model of this application is the mixed feature adversarial training method, which uses the feature extractor and the proportion discriminator to conduct a minimum game, that is, the proportion discriminator tries to determine what the specific mixing ratio is, and the goal of the feature extractor is to extract the most common features so that the proportion discriminator cannot know the specific value of the mixing ratio. This adversarial training can help the transfer learning model extract common features that are not affected by specific data sets, thereby improving the generalization ability of the transfer learning model on unknown data. Therefore, this application realizes the extraction of common features of the training set and the test set, so that only the training of the public data set can be used to analyze the images obtained in the actual power inspection scene, so as to optimize the performance of the transfer learning model and increase its generalization ability.

[0049] Specifically, Figure 2 As shown, the transfer learning model of the present application includes two feature extractors, two trained label classifiers, a gradient reversal layer and a scale discriminator.

[0050] (1) Gradient reversal layer

[0051] This application first introduces the standard back propagation process that does not involve the gradient reversal layer. Assume that there is a simple neural network, which includes an input layer, several hidden layers, and an output layer. The goal of the neural network is to minimize the loss function L for any parameter θ in the neural network. Its update rule is based on the gradient descent method, expressed as:

[0052]

[0053] Among them, α is the learning rate, is the gradient of the loss function L with respect to the parameter θ.

[0054] When the gradient reversal layer is introduced, this gradient update process will change. The gradient reversal layer is usually located between the feature extractor and the scale discriminator. Its main function is to modify the sign of the gradient during back propagation. Specifically, it multiplies by a negative constant. Let λ be this negative constant (let λ = -1), then the gradient update process is as follows:

[0055] During forward propagation, the gradient reversal layer does not change any data, that is, y = x, where x and y are the input and output of the gradient reversal layer, respectively. During backward propagation, let L be the loss function. If y is the output of the gradient reversal layer, then the gradient with respect to the input x (i.e., the gradient passed back to the gradient reversal layer) will be multiplied by λ, as shown in formula (4). Since λ is a negative number, this actually reverses the direction of the gradient. This process causes the feature extractor to update its weights in a direction that increases the loss of the proportional discriminator instead of in a direction that reduces the loss of the proportional discriminator, thereby achieving the purpose of adversarial training.

[0056]

[0057] (2) Feature Extraction Layers

[0058] The feature extractor is composed of a convolutional neural network module and the Transformer's Self-Attention module. The convolutional layer, batch normalization layer, pooling layer, ReLU activation function layer, and Dropout layer are used to build a basic convolutional neural network module. The specific information is as follows:

[0059] Table 1 Specific composition of convolutional neural network module

[0060]

[0061]

[0062] Convolutional Layer 1: First, the preprocessed training set and test set data are subjected to image convolution using a 5×5 filter (convolution kernel). This operation can capture local features in the image, such as edges, corners, etc. The first convolution layer converts the input RGB image (3 channels) into a 64-channel feature map. Each convolution operation can extract a different subset of features from the original image. For example, a specific convolution kernel may be used specifically to detect edges, while another may recognize specific textures.

[0063] The first batch normalization layer BatchNormalization1: Normalizes the primary feature map, which helps the transfer learning model converge faster and improves the robustness of the transfer learning model to small changes in input parameters. Normalization ensures that the feature map will not cause the gradient vanishing problem during the network training process due to excessive values ​​before passing through the ReLU activation function layer.

[0064] The first pooling layer Max Pooling1: Use a 2×2 pooling window to reduce the feature dimension, which not only reduces the consumption of computing resources, but also can extract more robust features to a certain extent. The first pooling layer retains important features by taking the maximum value of the region, while achieving spatial invariance of the features.

[0065] The first ReLU activation function layer ReLU Activation1: introduces nonlinear factors to the network. The ReLU activation function layer makes it difficult for the network to learn complex patterns. The first ReLU activation function layer helps the transfer learning model remove negative activations, so that only activated neurons can be kept in the network, improving sparsity and computational efficiency.

[0066] Convolutional Layer 2: Use a 5×5 filter (convolution kernel) to perform image convolution. The second convolution layer acts as a depth convolution to extract more detailed information.

[0067] The second batch normalization layer BatchNormalization2: Normalizes the feature map after re-extraction to speed up the convergence and alleviate the gradient vanishing problem during network training.

[0068] The first random dropout layer Dropout1: Randomly shut down some neurons to prevent overfitting of the transfer learning model and enhance the generalization ability of the transfer learning model. By reducing the dependency of neurons, the first random dropout layer can help the transfer learning model learn effective features in a variety of different network configurations.

[0069] The second pooling layer Max Pooling2: Use a 2×2 pooling window with a step size of 2 to reduce the feature dimension, reduce computing resource consumption, and take the regional maximum value to retain important features.

[0070] The second ReLU activation function layer ReLU Activation2: introduces nonlinear factors into the network to help the transfer learning model remove negative activations and improve sparsity and computational efficiency.

[0071] (3) Self-Attention Module

[0072] like Figure 3 As shown in Figure 1, Transformer effectively captures long-distance dependencies in the sequence through the Self-Attention module, which is crucial for understanding complex input structures (such as entire sentences or documents). After the CNN module extracts the detailed features in the image, this application uses the Self-Attention module to obtain global dependency information.

[0073] Input vector: First, the feature vector X passed through the second ReLU activation function layer ReLU Activation2, consists of several vectors, each of which is d-dimensional and represents an element in the sequence.

[0074] Linear transformation: For each element in the sequence, three different sets of weight vectors are calculated: query weight vector (Queries, Q), key weight vector (Keys, K) and value weight vector (Values, V). These weight vectors are obtained by combining the input feature vector with three learnable weight matrices W Q , W K and W V Multiplying together gives:

[0075] Q=XW Q (5)

[0076] K=XW K (6)

[0077] V=XW V (7)

[0078] Among them, W Q , W K and W V It is the parameter that the transfer learning model needs to learn.

[0079] Calculate attention scores: Each self-attention head calculates an attention score, which represents the influence of each position in the sequence on other positions. This is achieved by taking the dot product of the query weight vector and the key weight vector, and multiplying the key weight vector by its dimension d. k Scaling is done by the square root of:

[0080]

[0081] Among them, Attention(Q,K,V) represents the operation performed on the output of the previous layer when it passes through the Attention layer in the Self-Attention module.

[0082] Softmax normalization: The attention scores are normalized by the Softmax function so that their sum is 1. This gives a probability distribution representing the importance of each position to the current position.

[0083] Weighted value vector: Apply this probability distribution to the value weight vector, calculate the weighted sum, and get the final output of the Attention layer:

[0084] Output=Attention(Q,K,V) (9)

[0085] Multi-head attention: Use multi-head attention to expand the capabilities of the model. Each head uses a different weight matrix W Q , W K and W V , generate different attention points, and finally concatenate the outputs of all heads.

[0086] (4) Label Classifier

[0087] The label classifier is a fully connected neural network used for category prediction. The specific structure is shown in Table 2.

[0088] Table 2 Specific composition of label classifier

[0089]

[0090]

[0091] First fully connected layer Fully Connected Layer1: Each input node is connected to each node in the next layer.

[0092] The third batch normalization layer BatchNormalization3: is used to adjust the output of the previous layer so that its mean is 0 and its variance is 1. This helps speed up training and reduce the sensitivity of the transfer learning model to the initialization weights.

[0093] The third ReLU activation function layer ReLU Activation3: Linear rectification function (ReLU), a non-linear activation function, is used to increase the non-linear characteristics of the network so that the network can learn more complex patterns.

[0094] The second random dropout layer, Dropout2, is used to randomly close 50% of the nodes to prevent the transfer learning model from overfitting.

[0095] Fully Connected Layer 2: Each input node is connected to each node in the next layer. The number of input nodes in the second layer is 100.

[0096] The fourth batch normalization layer BatchNormalization4: is used to adjust the output of the previous layer so that its mean is 0 and its variance is 1. This helps speed up training and reduce the sensitivity of the transfer learning model to the initialization weights.

[0097] The fourth ReLU activation function layer ReLU Activation4: same as ReLUActivation3.

[0098] The third fully connected layer Fully ConnectedLayer3: Assuming that there are 10 types of images to be judged, the output is 10 nodes.

[0099] Log Softmax layer: Log Softmax: Usually used as the output layer for multi-classification problems, it converts the input value into logarithmic probability form, which is an effective form for calculating cross entropy loss.

[0100] The role of a label classifier is to predict the category of each input sample. In a neural network, this is usually achieved by applying a Softmax function after the last fully connected layer. The Softmax function converts the raw predictions output by the neural network, i.e. logits, into a probability distribution. Label classifiers are usually trained using a cross-entropy loss function, which measures the difference between the actual label and the model's prediction. For multi-classification problems, the formula for the cross-entropy loss function is:

[0101]

[0102] Here, M is the total number of categories, y iis the one-hot encoding of the true label of the i-th class, p i is the predicted probability of the i-th class. When the predicted probability p i When y is close to 1, i is also 1 (indicating the true label), the loss will tend to 0. If the predicted probability p i and the true label y i If not, the losses will increase.

[0103] (5) Ratio Discriminator

[0104] The ratio discriminator is also a fully connected neural network, which is used to discriminate the mixing ratio of input features. The specific structure is shown in Table 3.

[0105] Table 3 Specific composition of the ratio discriminator

[0106]

[0107]

[0108] Fourth fully connected layer Fully Connected Layer4: This is the first layer of the scale discriminator, which aims to linearly transform the input mixed features (assuming it is a 750-dimensional vector representing the mixed features) into a new 100-dimensional feature space. The fourth fully connected layer implements this linear transformation through the weight matrix and bias vector, which can capture the linear relationship between the input features.

[0109] Batch Normalization5: The purpose of this layer is to normalize the output of the previous fully connected layer, that is, to adjust the mean and variance of the output value to make its distribution more stable. This helps to accelerate the convergence speed of the transfer learning model training process, and at the same time can improve the generalization ability of the transfer learning model and prevent overfitting.

[0110] The fifth ReLU activation function layer ReLU Activation5: The purpose of the activation function is to introduce nonlinearity in the transfer learning model. Its formula is shown in formula (11), that is, when the input is less than 0, the output is 0, and when it is greater than 0, the input value is output. This can help the transfer learning model learn more complex feature representations.

[0111] Relu(x)=max(0,x) (11)

[0112] Among them, x represents the input of the Relu layer, that is, the output of the previous layer.

[0113] Fifth fully connected layer Fully Connected Layer5: This is the second fully connected layer in the scale discriminator, which further transforms the features and maps the 100-dimensional features to 50 dimensions. This layer continues to capture the linear relationship between features and provides input for subsequent layers.

[0114] Batch Normalization6: Similar to the previous batch normalization layer, this layer continues to normalize the features to maintain the stability of the data during training and improve the learning efficiency and performance of the transfer learning model.

[0115] The sixth ReLU activation function layer ReLU Activation6: This is the second ReLU activation function layer of the scale discriminator. Its function is the same as the first ReLU activation function layer. It continues to add necessary nonlinearity to the transfer learning model, which helps to process more complex feature relationships.

[0116] Fully Connected Layer6: This is the last layer of the ratio discriminator, whose task is to reduce the 50-dimensional feature vector to a single output value, the predicted mixing ratio. The output of this layer is a continuous value representing the predicted mixing ratio of the source domain (training set) and the target domain (test set) features.

[0117] The goal of the ratio discriminator is to determine the mixing ratio of the training set features to the test set features. In adversarial networks, this is usually achieved by training a ratio discriminator that uses mean squared error (MSE) to distinguish between two different data sets. The MSE loss function calculates the mean of the squared difference between the predicted value and the target value, and its formula is:

[0118]

[0119] Here, N is the total number of samples, y c is the target value of sample c, and is the value predicted by the model. In transfer learning and adversarial training scenarios, the feature extractor is trained to fool the scale discriminator into being unable to distinguish features from the training set or the test set, thereby forcing the feature extractor to capture more generalizable features.

[0120] During the training process, the training set features and the test set features are mixed in a certain ratio and then input into the ratio discriminator. By minimizing the loss, the ratio discriminator learns to distinguish the mixing ratio.

[0121]

[0122] Here, D represents the ratio discriminator, F(x) represents the output of the feature extractor, and when x is X train When x is X, it means it comes from the training set features. test , which means it comes from the test set features; λ represents the proportion of the mixture; L d is the loss function of the ratio discriminator; f MIX(F(X train ),F(X test );λ) is to mix the training set features and the test set features according to the λ ratio; l mse is the mean square error loss; D(f mix ) indicates that the mixed training set features and test set features are input into the ratio discriminator D for calculation.

[0123] The feature extractor and the ratio discriminator are jointly trained using the labeled data of the training set and the pseudo-labeled data of the test set, so that the features learned by the transfer learning model should be common between the source domain and the target domain, so that the transfer learning model can operate seamlessly on both domains. At the same time, the ratio discriminator continues to try to distinguish the mixing ratio, and is trained through the gradient reversal layer to confuse the ratio discriminator as much as possible.

[0124] Step 5: Analyze the data collected from actual power inspection scenarios through the trained transfer learning model to complete the drone industrial inspection.

[0125] After obtaining the trained transfer learning model through the above steps, a drone is used to collect images of actual power inspection scenes, and the trained transfer learning model is used to analyze the collected images to obtain accurate labels for the images, determine whether there are anomalies in the images, and what kind of anomalies exist, thereby completing the drone's industrial inspection.

[0126] In summary, this application expands the data set in data preprocessing and increases the generalization ability of the transfer learning model. For the feature extraction part of the transfer learning model, this application incorporates the self-attention module into the feature extraction, and the process of extracting the self-attention module is similar to the high-dimensional feature extraction part. In addition, in terms of adversarial learning strategy, this application adopts a hybrid feature adversarial learning strategy to learn a smoother and more continuous domain-invariant latent space, further improving the generalization ability of the model, thereby effectively improving the efficiency and accuracy of drone industrial inspections.

[0127] In some embodiments, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the transfer learning-based drone industrial inspection method.

[0128] In some embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the transfer learning-based drone industrial inspection method.

[0129] In some embodiments, the present invention also provides a computer device, including a processor, a memory, an input / output interface (I / O for short), a communication interface, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the transfer learning-based UAV industrial inspection method.

[0130] Wherein, the processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the unmanned aerial vehicle industrial inspection method based on transfer learning is implemented.

[0131] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A UAV industrial inspection method based on transfer learning, characterized in that: include: Obtain drone inspection data set and preprocess it; The drone inspection dataset includes a training set and a test set; Pre-train the label classifier based on the preprocessed training set to obtain a trained label classifier; The preprocessed test set is classified using the trained label classifier to obtain a classified test set; Inputting the preprocessed training set and the classified test set into the transfer learning model for mixed feature adversarial learning training to obtain a trained transfer learning model; the transfer learning model includes two feature extractors, two trained label classifiers, a gradient reversal layer, and a scale discriminator; The trained transfer learning model is used to analyze the data collected from actual power inspection scenarios and complete drone industrial inspections.

2. The UAV industrial inspection method based on transfer learning according to claim 1 is characterized in that: The obtaining of the drone inspection data set and preprocessing specifically include: After acquiring images of cracks on insulators of electric wires, images of flashovers on power lines, and images of normal power lines from a large-scale visual data set, labels are added to each image to obtain a training set; the labels include crack locations and flashover types; The test set is obtained by directly using drones to obtain images of wire insulator cracks, line flashover images, and normal power line images from actual power line inspection scenarios; The training set and test set are preprocessed by adding Gaussian noise and rotating the image, respectively, to obtain the preprocessed training set and test set.

3. The UAV industrial inspection method based on transfer learning according to claim 2 is characterized in that: The trained label classifier is used to classify the preprocessed test set to obtain a classified test set, which specifically includes: Use the trained label classifier to classify the preprocessed test set, and set the confidence threshold to 0.9; When the probability of the preprocessed test set being solved by the trained label classifier is greater than 0.9, a pseudo label corresponding to the preprocessed test set is generated to obtain a classified test set.

4. The UAV industrial inspection method based on transfer learning according to claim 3 is characterized in that: The preprocessed training set and the classified test set are input into the transfer learning model to perform mixed feature adversarial learning training to obtain a trained transfer learning model, specifically including: The preprocessed training set is input into the first feature extractor, and the classified test set is input into the second feature extractor, and the training set features and the test set features are output respectively; The training set features are input into the first trained label classifier, and the test set features are input into the second trained label classifier to predict the correct label. At the same time, the training set features and the test set features are mixed in a preset ratio and then input into the ratio discriminator through the gradient reversal layer to discriminate the source of the data; The minimum game is carried out through the first feature extractor, the second feature extractor and the ratio discriminator to complete the mixed feature adversarial learning training and obtain the trained transfer learning model.

5. The UAV industrial inspection method based on transfer learning according to claim 1 is characterized in that: The feature extractor is composed of a convolutional neural network module and a self-attention module; the convolutional neural network includes a first convolutional layer, a first batch normalization layer, a first pooling layer, a first ReLU activation function layer, a second convolutional layer, a second batch normalization layer, a first random drop layer, a second pooling layer and a second ReLU activation function layer connected in sequence.

6. The UAV industrial inspection method based on transfer learning according to claim 1 is characterized in that: The label classifier includes a first fully connected layer, a third batch normalization layer, a third ReLU activation function layer, a second random drop layer, a second fully connected layer, a fourth batch normalization layer, a fourth ReLU activation function layer, a third fully connected layer and a logarithmic Softmax layer, which are connected in sequence.

7. The UAV industrial inspection method based on transfer learning according to claim 1 is characterized in that: The scale discriminator includes a fourth fully connected layer, a fifth batch normalization layer, a fifth ReLU activation function layer, a fifth fully connected layer, a sixth batch normalization layer, a sixth ReLU activation function layer and a sixth fully connected layer, which are connected in sequence.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the UAV industrial inspection method based on transfer learning described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the UAV industrial inspection method based on transfer learning described in any one of claims 1 to 7 is implemented.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the UAV industrial inspection method based on transfer learning as described in any one of claims 1 to 7.

Images