A power inspection target detection method based on large model efficient parameter fine tuning

Abstract

The application relates to a power inspection target detection method based on efficient parameter fine-tuning of a large model, which comprises the following steps: acquiring a data set in a power inspection scene, and preprocessing the data set; dividing the preprocessed data set into a training set and a test set according to a proportion; taking a visual converter model as a basis, placing an additional fine-tuning module after each layer of the visual converter model, and constructing a power inspection target detection model based on the efficient parameter fine-tuning method of the large model; inputting the training set into the power inspection target detection model, updating the module parameters of the fine-tuning module through back propagation, and obtaining a trained power inspection target detection model; and inputting the test set into the trained power inspection target detection model, and obtaining a power inspection target detection result. The method can not only retain the strong generalization ability of the large model, but also be flexibly adjusted according to the specific application scene, so that the precision and efficiency of the power inspection target detection are improved.

Description

Technical Field

[0001] This application relates to the field of power line inspection target detection technology, and in particular to a power line inspection target detection method based on efficient parameter fine-tuning of a large model. Background Technology

[0002] Currently, in the field of power system monitoring, efficient parameter fine-tuning methods using large models can quickly train fine-tuned large models adaptable to various downstream scenarios, thus fully utilizing the powerful generalization ability of large models. However, this method also encounters some challenges in practical applications. First, power system data is often high-dimensional and large-scale, making it difficult for traditional algorithms to handle such data. Second, the power system environment is often complex and variable, making it difficult for a single model to perform well in various downstream tasks. Therefore, designing a robust model to adapt to different downstream tasks for various monitoring objects is also a challenge. Finally, the safety of the power system is paramount, and ensuring the safety and reliability of the model is also a problem that needs to be considered. Summary of the Invention

[0003] Therefore, it is necessary to provide a power line inspection target detection method that can retain the strong generalization ability of a large model and can be flexibly adjusted according to specific application scenarios, thereby improving detection accuracy and efficiency. Specifically, it is a power line inspection target detection method based on efficient parameter fine-tuning of a large model.

[0004] This invention provides a power line inspection target detection method based on efficient parameter fine-tuning of a large model, the method comprising:

[0005] S1: Obtain the dataset in the power inspection scenario, preprocess the dataset, and divide the preprocessed dataset into training set and test set according to the proportion;

[0006] S2: Based on the visual transformer model, an additional fine-tuning module is placed after each Transformer block to construct a power inspection target detection model based on a large model efficient parameter fine-tuning method; the fine-tuning module is used to downsample, upsample and regularize the prediction results of the Transformer block in sequence to fine-tune the power inspection target detection model.

[0007] S3: Input the training set into the power line inspection target detection model, update the module parameters of the fine-tuning module through backpropagation, and obtain the trained power line inspection target detection model;

[0008] S4: Input the test set into the trained power line inspection target detection model to obtain the power line inspection target detection results.

[0009] Preferably, the dataset for the power inspection scenario includes images from the power inspection scenario.

[0010] Preferably, the preprocessing of the dataset includes dividing the images into a set of image sequences.

[0011] Preferably, the visual converter model includes an embedding layer, a position encoding layer, a Transformer block, and an output header;

[0012] The embedding layer is used to perform graph embedding on each image in the image sequence to obtain an input vector;

[0013] The location encoding layer is used to perform location encoding on each image to obtain a location encoding representation of each image;

[0014] The Transformer block is used to map the input vector to different subspaces, perform attention calculation on the input vector in each subspace, add the calculation results of all subspaces to obtain the attention sum, convert the attention sum into a classification matrix, and perform a nonlinear transformation on the classification matrix to obtain the prediction result.

[0015] The output header is used to output the prediction result.

[0016] Preferably, the Transformer block includes a first normalization layer, a multi-head self-attention layer, a second normalization layer, and a feedforward neural network.

[0017] Preferably, the step of inputting the training set into the power line inspection target detection model and updating the module parameters of the fine-tuning module through backpropagation to obtain the trained power line inspection target detection model includes:

[0018] The original weights and bias terms of the power line inspection target detection model are frozen, so that the power line inspection target detection model only participates in forward propagation; the module parameters of the fine-tuning module are updated by backpropagation through the detection boxes, target detection box outputs and loss functions set in the training set until convergence or the maximum number of iterations is reached, so as to obtain the trained power line inspection target detection model.

[0019] Preferably, the process of obtaining the detection results of power line inspection targets includes:

[0020] Step 1: Initialize the position encoding matrix, denoted as P, with a size of n*d. model , where n is the length of the image sequence; d model Dimensions of the power line inspection target detection model;

[0021] Step 2: Multiply the position encoding matrix by each image in the image sequence to obtain the position encoding of each image;

[0022] Step 3: Multiply the input vector by multiple different linear transformation matrices to map the input vector to multiple different subspaces;

[0023] Step 4: In each subspace, initialize the attention weight matrix of the trained power line inspection target detection model, denoted as W. attention Its dimension is d model *d k d model d represents the dimension of the power line inspection target detection model. k The dimension of the key matrix in the attention mechanism

[0024] Step 5: Multiply the input vector by the query weight matrix, key weight matrix, and value weight matrix respectively to obtain the query matrix, key matrix, and value matrix;

[0025] Step 6: Perform a dot product operation between the query matrix and the key matrix to obtain the attention weights;

[0026] Step 7: Obtain the self-attention representation based on the attention weights and the value matrix;

[0027] Step 8: Repeat steps 4-7 until all subspaces have been traversed, and add the self-attention representations of the subspaces to obtain the total attention.

[0028] Step 9: Convert the sum of attention into a classification matrix through a fully connected layer; perform a nonlinear transformation on the classification matrix using an activation function to obtain the prediction result;

[0029] Step 10: The prediction results are passed through the fine-tuning module, and downsampling, upsampling and regularization are performed in sequence to fine-tune the power inspection target detection model and obtain the power inspection target detection result;

[0030] Step 11: Input the detection results of the power inspection target.

[0031] Preferably, the formulas for calculating the query matrix, key matrix, and value matrix are as follows:

[0032] Q = W q X;

[0033] K = W k X;

[0034] V = W v X;

[0035] Where Q is the query matrix, K is the key matrix, V is the value matrix, and W is the value matrix. q To query the weight matrix, W k W is the key weight matrix. v is the value weight matrix; X is the input vector.

[0036] The preferred formula for calculating self-attention representation is:

[0037] SA(X) = Softmax(Score) * V;

[0038]

[0039] Where SA(X) is the self-attention representation, Softmax(·) is the Softmax function, Score is the attention weight, Q is the query matrix, K is the key matrix, V is the value matrix, and d k denoted as the dimension of the key matrix.

[0040] Preferably, the fine-tuning module includes a downsampling layer, an upsampling layer, and a degradation layer; the dimension of the downsampling layer is n. up *n down ;n up n represents the dimension of the prediction result. down The dimension of the downsampled layer is n; the dimension of the upsampled layer is n. down *n up ;

[0041] The working process of the fine-tuning module includes:

[0042] Step 1: Downsample the prediction result using the downsampling layer to reduce the dimensionality of the prediction result to n. down dimension;

[0043] Step 2: Upsample the downsampled prediction result through the upsampling layer to restore the dimension of the downsampled prediction result to the original dimension;

[0044] Step 3: Regularize the predicted results restored to the original dimension using the linearly changing degradation rate in the degradation layer, and fine-tune the power line inspection target detection model; the formula for calculating the linear change of the degradation rate is:

[0045]

[0046] Where, d l d represents the degradation rate of the l-th layer of the power line inspection target detection model, and d0 represents the degradation rate of the 0-th layer of the power line inspection target detection model. L This represents the degradation rate of the Lth layer of the power line inspection target detection model, where L is the total number of layers in the power line inspection target detection model.

[0047] Beneficial effects: This method is based on the visual transformer model and places an additional fine-tuning module after each Transformer block to construct a power line inspection target detection model based on a large model efficient parameter fine-tuning method. It can retain the strong generalization ability of the large model and can be flexibly adjusted according to specific application scenarios, thereby improving the accuracy and efficiency of power line inspection target detection. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of a power line inspection target detection method based on efficient parameter fine-tuning of a large model, according to an embodiment of this application. Detailed Implementation

[0050] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0051] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0052] like Figure 1 As shown in the figure, this embodiment provides a power line inspection target detection method based on efficient parameter fine-tuning of a large model. The method includes:

[0053] S1: Obtain the dataset for the power inspection scenario, preprocess the dataset, and divide the preprocessed dataset into training set and test set according to the ratio.

[0054] Furthermore, the dataset for the power inspection scenario includes images from the power inspection scenario.

[0055] Furthermore, the preprocessing includes dividing the image into a sequence of pictures.

[0056] In this embodiment, the ratio of the training set to the test set is 8:2.

[0057] S2: Based on the visual transformer model, an additional fine-tuning module is placed after each Transformer block to construct a power inspection target detection model based on a large model efficient parameter fine-tuning method; the fine-tuning module is used to downsample, upsample and regularize the prediction results of the Transformer block in sequence to fine-tune the power inspection target detection model.

[0058] Specifically, the visual converter model includes an embedding layer, a position encoding layer, a Transformer block, and an output header;

[0059] The embedding layer is used to perform graph embedding on each image in the image sequence to obtain an input vector;

[0060] The location encoding layer is used to perform location encoding on each image to obtain a location encoding representation of each image;

[0061] The Transformer block is used to map the input vector to different subspaces, perform attention calculation on the input vector in each subspace, add the calculation results of all subspaces to obtain the attention sum, convert the attention sum into a classification matrix, and perform a nonlinear transformation on the classification matrix to obtain the prediction result.

[0062] Furthermore, the Transformer block includes a first normalization layer, a multi-head self-attention layer, a second normalization layer, and a feedforward neural network; the multi-head self-attention layer is used to capture the correlation between different positions in the input sequence and learn the global features of the sequence; the feedforward neural network is used to enhance the linearity of the model, thereby improving the expressive power of the model.

[0063] The output header is used to output the prediction result.

[0064] The fine-tuning module can fine-tune the model by introducing a small number of additional parameters, enabling the model to adapt to different downstream tasks while maintaining the strong generalization performance of the large model.

[0065] S3: Input the training set into the power line inspection target detection model, update the module parameters of the fine-tuning module through backpropagation, and obtain the trained power line inspection target detection model.

[0066] Specifically, this includes: freezing the original weights and bias terms of the power line inspection target detection model so that the power line inspection target detection model only participates in forward propagation; updating the module parameters of the fine-tuning module through backpropagation using the detection boxes, target detection box outputs, and loss functions set in the training set, until convergence or the maximum number of iterations is reached, to obtain the trained power line inspection target detection model.

[0067] S4: Input the test set into the trained power line inspection target detection model to obtain the power line inspection target detection results.

[0068] Specifically, the process of obtaining the detection results of power line inspection targets includes:

[0069] Step 1: Initialize the position encoding matrix, denoted as P, with a size of n*d. model , where n is the length of the image sequence; d model Dimensions of the power line inspection target detection model;

[0070] Step 2: Multiply the position encoding matrix by each image in the image sequence to obtain the position encoding of each image;

[0071] Step 3: Multiply the input vector by multiple different linear transformation matrices to map the input vector to multiple different subspaces;

[0072] Step 4: In each subspace, initialize the attention weight matrix of the trained power line inspection target detection model, denoted as W. attention Its dimension is d model *d k d model d represents the dimension of the power line inspection target detection model. k The dimension of the key matrix in the attention mechanism

[0073] Step 5: Multiply the input vector by the query weight matrix, key weight matrix, and value weight matrix respectively to obtain the query matrix, key matrix, and value matrix; the formulas for calculating the query matrix, key matrix, and value matrix are as follows:

[0074] Q = W q X;

[0075] K = W k X;

[0076] V = W v X;

[0077] Where Q is the query matrix, K is the key matrix, V is the value matrix, and W is the value matrix. q To query the weight matrix, W kW is the key weight matrix. v is the value weight matrix; X is the input vector.

[0078] Step 6: Perform a dot product operation between the query matrix and the key matrix to obtain the attention weights;

[0079] To make the self-attention representation more stable during network training and avoid gradient explosion or other issues, the attention weights are calculated by dividing the dot product result by the dimension of the key matrix:

[0080]

[0081] Where Score is the attention weight, Q is the query matrix, K is the key matrix, and d k denoted as the dimension of the key matrix.

[0082] In this embodiment, the query matrix and the key matrix can also use other similarity measurement methods to calculate attention weights.

[0083] Step 7: Obtain the self-attention representation based on the attention weights and the value matrix;

[0084] Specifically, the attention weights are transformed into a probability distribution from 0 to 1 using the Softmax function, and all probabilities are summed to 1. The result is then multiplied by the value matrix to obtain the self-attention representation. The attention weights maintain the weights of relevant pixel values ​​in the image sequence without reducing the weights of other pixel values. The formula for calculating the self-attention representation is as follows:

[0085] SA(X) = Softmax(Score) * V;

[0086] Where SA(X) is the self-attention representation, Softmax(·) is the Softmax function, Score is the attention weight, Q is the query matrix, K is the key matrix, V is the value matrix, and d k denoted as the dimension of the key matrix.

[0087] Step 8: Repeat steps 4-7 until all subspaces have been traversed, and add the self-attention representations of the subspaces to obtain the total attention.

[0088] Step 9: Convert the sum of attention into a classification matrix through a fully connected layer; perform a nonlinear transformation on the classification matrix using an activation function to obtain the prediction result;

[0089] Step 10: The prediction results are passed through the fine-tuning module, and downsampling, upsampling and regularization are performed in sequence to fine-tune the power inspection target detection model and obtain the power inspection target detection result;

[0090] Furthermore, the fine-tuning module includes a downsampling layer, an upsampling layer, and a degradation layer; the dimension of the downsampling layer is n. up *n down ;n up n represents the dimension of the prediction result. down The dimension of the downsampled layer is n; the dimension of the upsampled layer is n. down *n up ;

[0091] The working process of the fine-tuning module includes:

[0092] Step 1: Downsample the prediction result using the downsampling layer to reduce the dimensionality of the prediction result to n. down dimension;

[0093] Downsampling the prediction results can reduce the number of features and thus reduce computational complexity, that is, filter out some minor features and retain key features.

[0094] Step 2: Upsample the downsampled prediction result through the upsampling layer to restore the dimension of the downsampled prediction result to the original dimension;

[0095] Upsampling the prediction results after downsampling can increase the number of features, thereby restoring the resolution of the original data. At the same time, the upsampling process will also fill in the information lost during downsampling.

[0096] Step 3: Regularize the predicted results restored to the original dimension using the linearly changing degradation rate in the degradation layer, and fine-tune the power line inspection target detection model; the formula for calculating the linear change of the degradation rate is:

[0097]

[0098] Where, d l d represents the degradation rate of the l-th layer of the power line inspection target detection model, and d0 represents the degradation rate of the 0-th layer of the power line inspection target detection model. L This represents the degradation rate of the Lth layer of the power line inspection target detection model, where L is the total number of layers in the power line inspection target detection model.

[0099] Compared to the constant degradation rate used in conventional degradation layers, the degradation layer in this embodiment uses a larger degradation rate in the shallow layers of the network, and the degradation rate decreases linearly with the increase of the number of network layers. In the shallow layers of the network, the information density is relatively sparse, and a larger degradation rate can extract feature information more efficiently; while in the deep layers of the network, the information density is relatively dense. A smaller degradation rate allows it to retain more feature information.

[0100] Step 11: Input the detection results of the power inspection target.

[0101] The method provided in this embodiment is based on a visual transformer model, and an additional fine-tuning module is placed after each Transformer block to construct a power line inspection target detection model based on a large model efficient parameter fine-tuning method. The original large model weights are frozen, and by introducing additional fine-tuning modules and a redesigned degradation layer, the strong generalization ability of the large model can be preserved, and it can be flexibly adjusted according to specific application scenarios, thereby improving the accuracy and efficiency of power line inspection target detection.

[0102] The method provided in this embodiment introduces an efficient large model fine-tuning method into the field of power inspection target detection. By freezing the original weights of the large model and only iterating the parameters in the fine-tuning module, the usage threshold of the large model is reduced, enabling various visual large models to be widely used in power scenarios.

[0103] Furthermore, the fine-tuning module has been improved by introducing a degradation layer with a linearly varying degradation rate, which enables large models to exhibit stronger generalization performance and quickly and effectively adapt to downstream tasks of various detection objects.

[0104] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A power line inspection target detection method based on efficient parameter fine-tuning of a large model, characterized in that, include: S1: Obtain the dataset in the power inspection scenario, preprocess the dataset, and divide the preprocessed dataset into training set and test set according to the proportion; The dataset for the power inspection scenario includes images from the power inspection scenario; preprocessing includes dividing the images into a set of image sequences; S2: Based on the visual transformer model, an additional fine-tuning module is placed after each Transformer block to construct a power inspection target detection model based on a large model efficient parameter fine-tuning method; the fine-tuning module is used to downsample, upsample and regularize the prediction results of the Transformer block in sequence to fine-tune the power inspection target detection model. The visual converter model includes an embedding layer, a position encoding layer, a Transformer block, and an output header; The embedding layer is used to perform graph embedding on each image in the image sequence to obtain an input vector; The location encoding layer is used to perform location encoding on each image to obtain a location encoding representation of each image; The Transformer block is used to map the input vector to different subspaces, perform attention calculation on the input vector in each subspace, add the calculation results of all subspaces to obtain the attention sum, convert the attention sum into a classification matrix, and perform a nonlinear transformation on the classification matrix to obtain the prediction result. The output header is used to output the prediction result; The fine-tuning module includes a downsampling layer, an upsampling layer, and a degradation layer; the dimension of the downsampling layer is [missing information]. n up n down ; n up For the dimensions of the prediction results, n down The dimension of the downsampled layer is ; the dimension of the upsampled layer is . n down n up ; The working process of the fine-tuning module includes: Step 1: Downsample the prediction result using the downsampling layer to reduce the dimensionality of the prediction result to... n down dimension; Step 2: Upsample the downsampled prediction result through the upsampling layer to restore the dimension of the downsampled prediction result to the original dimension; Step 3: Regularize the predicted results restored to the original dimension using the linearly changing degradation rate in the degradation layer, and fine-tune the power line inspection target detection model; the formula for calculating the linear change of the degradation rate is: ； in, d l The power line inspection target detection model is represented by the first... l Layer degradation rate, d 0 represents the degradation rate of layer 0 in the power line inspection target detection model. d L The power line inspection target detection model is represented by the first... L Layer degradation rate, L This represents the total number of layers in the power line inspection target detection model. S3: Input the training set into the power line inspection target detection model, update the module parameters of the fine-tuning module through backpropagation, and obtain the trained power line inspection target detection model; S4: Input the test set into the trained power line inspection target detection model to obtain the power line inspection target detection results.

2. The power line inspection target detection method based on efficient parameter fine-tuning of a large model according to claim 1, characterized in that, The Transformer block includes a first normalization layer, a multi-head self-attention layer, a second normalization layer, and a feedforward neural network.

3. The power line inspection target detection method based on efficient parameter fine-tuning of a large model according to claim 1, characterized in that, The step of inputting the training set into the power line inspection target detection model and updating the module parameters of the fine-tuning module through backpropagation to obtain the trained power line inspection target detection model includes: The original weights and bias terms of the power line inspection target detection model are frozen, so that the power line inspection target detection model only participates in forward propagation; the module parameters of the fine-tuning module are updated by backpropagation through the detection boxes, target detection box outputs and loss functions set in the training set until convergence or the maximum number of iterations is reached, so as to obtain the trained power line inspection target detection model.

4. The power line inspection target detection method based on efficient parameter fine-tuning of a large model according to claim 1, characterized in that, The process of obtaining the detection results of power line inspection targets includes: Step 1: Initialize the position encoding matrix, the dimension of which is denoted as . P Its dimension size is n d model , n The length of the image sequence; d model Dimensions of the power line inspection target detection model; Step 2: Multiply the position encoding matrix by each image in the image sequence to obtain the position encoding of each image; Step 3: Multiply the input vector by multiple different linear transformation matrices to map the input vector to multiple different subspaces; Step 4: In each subspace, initialize the attention weight matrix of the trained power line inspection target detection model, denoted as […]. W attention Its dimension size is d model d k , d model The dimensions of the power line inspection target detection model. d k The dimension of the key matrix in the attention mechanism Step 5: Multiply the input vector by the query weight matrix, key weight matrix, and value weight matrix respectively to obtain the query matrix, key matrix, and value matrix; Step 6: Perform a dot product operation between the query matrix and the key matrix to obtain the attention weights; Step 7: Obtain the self-attention representation based on the attention weights and the value matrix; Step 8: Repeat steps 4-7 until all subspaces have been traversed, and add the self-attention representations of the subspaces to obtain the total attention. Step 9: Convert the sum of attention into a classification matrix through a fully connected layer; perform a nonlinear transformation on the classification matrix using an activation function to obtain the prediction result; Step 10: The prediction results are passed through the fine-tuning module, and downsampling, upsampling and regularization are performed in sequence to fine-tune the power inspection target detection model and obtain the power inspection target detection result; Step 11: Output the detection results of the power inspection target.

5. The power line inspection target detection method based on efficient parameter fine-tuning of a large model according to claim 4, characterized in that, The formulas for calculating the query matrix, key matrix, and value matrix are as follows: ； ； ； in, Q For querying the matrix, K The key matrix, V For value matrices, W q To query the weight matrix, W k The key weight matrix is... W v Value weight matrix; X The input vector.

6. The power line inspection target detection method based on efficient parameter fine-tuning of a large model according to claim 5, characterized in that, The formula for calculating self-attention is: ； ； in, SA ( X () represents self-attention. Softmax (·)for Softmax function, Score For attention weights, Q For querying the matrix, K The key matrix, V For value matrices, d k denoted as the dimension of the key matrix.

Images