A method for detecting lightning ablation of metal composite materials

By introducing a Transformer-based detection method in machine vision ablation detection, the problems of computational complexity and ablation feature extraction difficulty in complex environments are solved, and efficient and accurate ablation detection is achieved.

CN119228792BActive Publication Date: 2025-02-07INST OF ENERGY HEFEI COMPREHENSIVE NAT SCI CENT (ANHUI ENERGY LAB) +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411730244.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-29
Publication Date
2025-02-07
Estimated Expiration
2044-11-29

AI Technical Summary

Technical Problem

The existing machine vision-based material ablation detection method requires complex data processing in complex environments, which increases the computational complexity and is difficult to extract ablation features.

Method used

Using a machine vision detection method based on Transformer, image features are extracted through convolutional layers and converted into one-dimensional data, input into Transformer encoder in combination with position encoding, learn related features, and predict ablation phenomena and bounding boxes through the decoder.

Benefits of technology

It realizes automatic, efficient and accurate identification of ablation phenomena in complex working conditions, reduces the impact of noise, and improves calculation efficiency and prediction accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119228792B_ABST
    Figure CN119228792B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of lightning ablation material detection, and in particular to a method for detecting lightning ablation of metal composite materials. The technical solution includes the following steps: obtaining a data set of lightning ablation images of metal composite materials to be detected, preprocessing the image data, presetting a bounding box, and learning a two-dimensional representation of the metal composite material ablation image data through a predefined convolution layer, and then stretching it into one-dimensional data. The Transformer-based machine vision detection method introduced in the present invention can automatically, efficiently and accurately identify ablation phenomena, effectively reduce the impact of noise, effectively and selectively capture global information, improve computational efficiency, strengthen the capture of global information while effectively reducing the amount of data calculation, and improve the accuracy of prediction and computational efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of lightning ablation material detection, and in particular to a lightning ablation detection method for metal composite materials. Background Art

[0002] Lightning strikes are an important cause of power grid failures. One of the effects of lightning strikes on power grids is that overhead ground wires will be damaged by ablation or even broken under the action of lightning arcs. The aluminum layer of overhead aluminum-clad steel ground wires that have been in operation for a long time will melt under the action of lightning strikes, and the steel core will be exposed. Long-term corrosion will greatly reduce the strength of the ground wire, which may cause line breakage accidents, endangering equipment, power grids and even personal safety. Therefore, ablation detection of metal composite materials is crucial in ensuring the safe and stable operation of power grids. Traditional ablation detection methods mainly rely on manual labor, but methods that rely on manual subjective judgment are inefficient, labor-intensive, and have high risk factors.

[0003] With the rapid development of computer and machine vision technology, engineering material detection technology based on machine vision has become a research hotspot. However, some projects, especially power grid lines, are usually built in remote geographical environments, and their working environment is extremely complex, especially mountainous lines that are prone to lightning strikes. These factors put forward additional requirements for detection technology.

[0004] The latest technology of material ablation detection based on machine vision mainly extracts features through deep convolutional neural networks. When applied to some complex working environments, due to the presence of many noise and interference sources, such as overall high-temperature oxidation, light changes, etc., the material ablation detection process needs to go through more complex data processing and denoising before subsequent detection can be carried out, which greatly increases the computational complexity. At the same time, since the ablation features in the sampled images are large or small and irregular in shape, it also makes it difficult to extract the features of material ablation.

[0005] Therefore, the present application proposes a method for detecting lightning ablation of metal composite materials. Summary of the invention

[0006] The purpose of the present invention is to propose a method for detecting lightning ablation of metal composite materials in order to address the problem that the existing material ablation detection process based on machine vision in the background technology requires more complex data processing and denoising methods before subsequent detection can be carried out, which increases the computational complexity.

[0007] The technical solution of the present invention is a method for detecting lightning ablation of metal composite materials, comprising the following steps:

[0008] Step S1: obtaining a lightning ablation image dataset of a metal composite material to be detected, preprocessing the image data, presetting a bounding box, and learning a two-dimensional representation of the metal composite material ablation image data through a predefined convolution layer, and then stretching it into one-dimensional data;

[0009] Step S2: The data converted into a one-dimensional representation of the ablation image after feature extraction is sent to the Transformer encoder together with the position code, so that the network can further learn relevant features. After deep neural network training, the obtained parameters are used by the decoder to learn the ablation phenomenon to be detected from the features;

[0010] Step S3: Perform bipartite graph matching on the position and label of the feature learned in the previous step with the position and label of the actual data set, and calculate the loss function of the label and the loss function of the position based on the matching results;

[0011] Step S4: Use the learned model to predict the bounding box of the ablation image to be identified, determine whether ablation exists in the image and its location, and obtain a picture with an ablation range box.

[0012] Optionally, the step S1 specifically includes:

[0013] Taking photos of metal composite materials with ablation phenomenon, developing an image dataset of metal composite materials with ablation phenomenon, extracting ablation feature maps of the original images through predefined convolutional layers, and setting a bounding box of ablation contained in the predicted image;

[0014] A rotation-adaptive bounding box is used, and the position parameters of the rotation bounding box are defined as a six-tuple ,in is the center point of the bounding box, is the width of the bounding box, is the height of the bounding box, is the rotation angle of the bounding box relative to the vertical line, Indicates whether there is ablation in the bounding box;

[0015] Use k-means clustering method to select A preset bounding box size, which is used to determine the size of the bounding box during detection;

[0016] The two-dimensional image after the convolution layer is stretched into one-dimensional data through the fully connected layer for the next step of processing.

[0017] Optionally, the step S2 specifically includes:

[0018] Stretch the two-dimensional data represented by the convolutional layer into one-dimensional data After that, the Transformer can meet the requirements of the data to be processed. At this time, the position code is extracted from it and the position code and data are combined. Input it into N encoding modules of the same composition to obtain a matrix that integrates all the attention head information, and then input it into the same N decoding modules of the same composition to obtain a The tensor is then fed into two feed-forward neural networks (FFNs) to determine whether there is ablation and the bounding box. is the training batch size.

[0019] Optionally, expand the trigonometric functions to dimensional vector as the representation of input information, different channels have different wavelengths;

[0020] in, For the channel, is the total number of channels, is the location information, is a coordinate in the feature map, Calculated separately.

[0021] Optionally, in the encoding process, N identical encoding modules are used, and the input of each encoding module is The image features and position encoding are outputted as feature matrices of the same dimension. Each encoding module consists of a multi-head self-attention layer and a feedforward neural network. is the height of the image feature, is the width of the image feature, is the training batch size.

[0022] Optionally, a variable multi-head self-attention mechanism is used. The calculation process of the multi-head self-attention layer is: the feature matrix and position encoding After adding, we get the query matrix , the bond matrix , value matrix That is the feature matrix , and the matrix containing self-attention information is calculated from these three matrices , The calculation formula of the matrix is:

[0023] in, The query matrix The weight matrix obtained after a linear layer is is the aggregation matrix of multiple groups of weight matrices, is the position encoding matrix, Value Weight , From the query matrix The calculated position offset is passed through the variable multi-head self-attention layer to convert the residual block Add to the resulting matrix , and then normalize it, and then send the obtained matrix into the feedforward neural network, which consists of a fully connected layer, a Relu activation layer, a Dropout layer and a fully connected layer, and then pass the residual connection and normalization. This is the role of an encoding module. After passing through N encoding modules, the result is sent to the decoder.

[0024] Optionally, in the decoding process, N identical modules are also used, where each module is composed of a variable multi-head self-attention layer MS1, a residual normalization layer, a variable multi-head self-attention layer MS2, a residual normalization layer, a feedforward neural network layer, and a residual normalization layer from bottom to top, where the residual normalization layer is the same as the feedforward neural network layer and the residual normalization layer and feedforward neural network layer of the encoder, while the input of the multi-head self-attention layer is different from that of the encoder, which can be specifically expressed as: the input of MS1 is a position matrix that is initially all 0 and the category query matrix Add the query matrix to MS1 and key matrix , and the value matrix is ​​directly given by The result is obtained by normalizing the residual layer and then comparing it with the category query matrix Add as the query matrix of MS2 , and the value matrix of MS2 The output of the encoder Directly obtain the bond matrix of MS2 The output of the encoder With position encoding Add together to get, where the category query matrix It is also a learnable matrix, and the output of MS2 is used as the new position matrix After passing through a feedforward neural network layer and a residual normalization layer, it is used as the output of the entire decoder module.

[0025] Optionally, the output of the decoder is sent to two different feedforward neural networks FFN1 and FFN2, where FFN1 is a category prediction network. This network can predict multiple categories, but only two categories need to be predicted as needed, 0 for no ablation and 1 for ablation. FFN1 is composed of a fully connected layer, and FFN2 is a bounding box prediction network. FFN2 is composed of three fully connected layers. The final output is a data of size 6, and the center point, height, width, and angle of the bounding box are these five data.

[0026] Optionally, the step S3 specifically includes:

[0027] An end-to-end loss calculation method is used, that is, the difference between the categories and bounding boxes in the training set and the final predicted results is directly calculated, and the loss function is determined based on this. The binary matching algorithm is first used to select the bounding box corresponding to the label from the predicted bounding box, and the label loss of all bounding boxes is calculated in turn. and bounding box loss , and modify the various matrix parameters in the Transformer structure according to the loss;

[0028] To calculate the loss function, we first need to use the bipartite graph matching algorithm to find the text box corresponding to the label, so that the final prediction result only calculates the loss with the matched true value. The formula for bipartite value matching is:

[0029] in is a true value, is the predicted value, It is a bipartite graph matching algorithm. The purpose of this formula is to find a predicted value that minimizes the bipartite graph matching loss between it and the true value. After finding the bounding box corresponding to the label, the loss of this method consists of two parts, namely the classification loss With bounding box error loss , the total loss function is:

[0030] The classification loss That is, whether it can be judged that the image has ablation phenomenon, and the bounding box error loss That is, the difference between the center point, height, width, and angle of the predicted bounding box and the real bounding box is calculated. The formula is expressed as:

[0031] in, They are the center point coordinates, width, height, and angle of the bounding box respectively.

[0032] Optionally, in the step S4, it specifically includes:

[0033] After the training is completed, the image to be tested is preprocessed and input into the model. The model will automatically predict several bounding boxes and label probabilities. If the probability of the label exceeds the designed threshold, the result is directly output. If the label probability does not exceed, the bounding box is considered to have no label and the bounding box is not displayed.

[0034] Compared with the prior art, the present invention has the following beneficial technical effects:

[0035] (1) When the ablation detection model in the present invention is applied to the ablation detection of metal composite materials under some complex working conditions, the introduced Transformer-based machine vision detection method can automatically, efficiently and accurately identify the ablation phenomenon, effectively reducing the influence of noise.

[0036] (2) The present invention uses a variable multi-head self-attention mechanism that can effectively and selectively capture global information, avoiding the problem of rapid increase in computational complexity when processing long sequence data, and effectively improving computational efficiency.

[0037] (3) The present invention designs an adaptive and rotatable bounding box, which can minimize the existence of irrelevant data in the bounding box, effectively reduce the amount of data calculation while enhancing the capture of global information, and improve the accuracy of prediction and the efficiency of calculation.

[0038] The Transformer-based machine vision detection method introduced in the present invention can automatically, efficiently and accurately identify ablation phenomena, effectively reduce the impact of noise, effectively and selectively capture global information, improve computing efficiency, enhance the capture of global information while effectively reducing the amount of data calculation, and improve the accuracy of prediction and computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a model structure diagram of a metal composite material ablation detection method based on Transformer according to an embodiment of the present application;

[0040] Figure 2 The encoder and decoder structures and calculation flow chart of the embodiment of the present application;

[0041] Figure 3 This is a comparison chart of the loss curves of the embodiment method of this application, yolov7 and spiking-yolo methods during training. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0043] Example

[0044] refer to Figure 1-Figure 2This embodiment provides a method for detecting ablation of a metal composite material. The metal composite material with ablation phenomenon is photographed, an image data set of the metal composite material with ablation phenomenon is prepared, ablation feature map of the original image is extracted through a predefined convolution layer, and a bounding box of ablation contained in the image to be predicted is set. In order to reduce the area of ​​pixels not related to the ablation phenomenon, this method uses a bounding box with a rotational adaptive size. The position parameter of the rotational bounding box can be defined as a six-tuple ,in is the center point of the bounding box, is the width of the bounding box, is the height of the bounding box, is the rotation angle of the bounding box relative to the vertical line, Indicates whether there is ablation in the bounding box. At the same time, the k-means clustering method is used to select A preset bounding box size is used to determine the size of the bounding box during detection. Then the two-dimensional image after the convolution layer is stretched into one-dimensional data through the fully connected layer for the next step of processing.

[0045] During the convolution process, the original image dimensions are generally: , its feature map is obtained through the residual convolutional network, and its feature map dimension is , generally, , , after compressing the number of channels through the convolution layer, a new feature map is obtained, whose dimension is After compressing it into one-dimensional data, its spatial dimension is compressed to one dimension, so its dimension is , is the height of the image feature, is the width of the image feature, is the training batch size.

[0046] Stretch the two-dimensional data represented by the convolutional layer into one-dimensional data In order to make the network perceive the position information of different inputs and balance the problem of large inputs and the periodicity of trigonometric functions, the trigonometric functions are expanded to dimensional vector as the representation of input information, and different channels have different wavelengths.

[0047] in, For the channel, is the total number of channels, is the location information, is a coordinate in the feature map, Calculated separately.

[0048] In the encoding process, N identical encoding modules are used, and the input of each encoding module is The image features and position encoding are outputted in the same dimension as the feature matrix. Each encoding module consists of a multi-head self-attention layer and a feedforward neural network.

[0049] This embodiment adopts a variable multi-head self-attention mechanism. It no longer uses global features as key values, but selects multiple sampling points at each position to calculate and learn image relationships. This method can effectively reduce the complexity of calculation and speed up training. The specific calculation process of the multi-head self-attention layer is: feature matrix and position encoding After adding, we get the query matrix , the bond matrix , value matrix That is the feature matrix , and the matrix containing self-attention information is calculated from these three matrices , The calculation formula of the matrix is:

[0050] in, The query matrix The weight matrix obtained after a linear layer is is the aggregation matrix of multiple groups of weight matrices, is the position encoding matrix, Value Weight , From the query matrix The calculated position offset is passed through the variable multi-head self-attention layer to convert the residual block Add to the resulting matrix , and then normalize it, so that the characteristics of the initial data can be retained, effectively preventing the neural network from degenerating during the training process. After normalization, the training speed can also be accelerated and the training stability can be improved. Then the obtained matrix is ​​sent to the feedforward neural network, which consists of a fully connected layer, a Relu activation layer, a Dropout layer and a fully connected layer, and then through residual connection and normalization. This is the role of an encoding module. After passing through N encoding modules, the result is sent to the decoder.

[0051] In the decoding process, N identical modules are also used, where each module consists of a variable multi-head self-attention layer MS1, a residual normalization layer, a variable multi-head self-attention layer MS2, a residual normalization layer, a feedforward neural network layer, and a residual normalization layer from bottom to top. Among them, the residual normalization layer is the same as the feedforward neural network layer and the residual normalization layer and feedforward neural network layer of the encoder, while the input of the multi-head self-attention layer is different from that of the encoder, which can be specifically expressed as follows: The input of MS1 is initially all 0 with a dimension of The position matrix With dimension The category query matrix Add as the query matrix of MS1 and key matrix , and the value matrix is ​​directly given by The result is obtained by normalizing the residual layer and then comparing it with the category query matrix Add as the query matrix of MS2 , and the value matrix of MS2 The output of the encoder Directly obtain the bond matrix of MS2 The output of the encoder With position encoding Add together to get, where the category query matrix It is also a learnable matrix. The output of MS2 is used as the new position matrix Through the feedforward neural network layer and the residual normalization layer, it is used as the output of the entire decoder module.

[0052] The size of the decoder is The output is sent to two different feedforward neural networks FFN1 and FFN2. FFN1 is a category prediction network. This network can predict multiple categories, but only two categories need to be predicted according to needs. 0 indicates no ablation and 1 indicates ablation. FFN1 is composed of a fully connected layer. FFN2 is a bounding box prediction network. FFN2 consists of three fully connected layers. The final output is a data of size 6. The center point, height, width, and angle of the bounding box are these five data.

[0053] This method uses an end-to-end loss calculation method, that is, directly calculating the difference between the categories and bounding boxes in the training set and the final predicted results, and using this as the basis to determine the loss function. First, a binary matching algorithm is used to select the bounding box corresponding to the label from the predicted bounding box, and the label loss of all bounding boxes is calculated in turn. and bounding box loss , and modify the various matrix parameters in the Transformer structure according to the loss.

[0054] Furthermore, to calculate the loss function, we first need to find the text box corresponding to the label through the bipartite graph matching algorithm, so that the final prediction result only calculates the loss with the matched true value. The formula for bipartite value matching is:

[0055] in is a true value, is the predicted value, It is a bipartite graph matching algorithm. The purpose of this formula is to find a predicted value that minimizes the bipartite graph matching loss between it and the true value. After finding the bounding box corresponding to the label, the loss of this method consists of two parts, namely the classification loss With bounding box error loss , the total loss function is:

[0056] The classification loss That is, whether it can be judged that the image has ablation phenomenon, and the bounding box error loss That is, the difference between the center point, height, width, and angle of the predicted bounding box and the real bounding box is calculated. The formula is expressed as:

[0057] in, They are the center point coordinates, width, height, and angle of the bounding box respectively.

[0058] After the training is completed, the image to be tested is preprocessed and input into the model. The model will automatically predict several bounding boxes and label probabilities. If the probability of the label exceeds the designed threshold, the result is directly output. If the label probability does not exceed, the bounding box is considered to have no label and the bounding box is not displayed.

[0059] In this embodiment, an overhead ground wire of a power grid in a southwestern region of China is selected as the research object, and the ablation images thereof are collected as a data set. There are a total of 1,100 image data with ablation phenomena, based on which the Transformer-based metal composite material ablation detection method proposed in the present invention is verified.

[0060] In this embodiment, in order to more accurately process the crack features in the image, we set the image to be input to a size of 640*640.

[0061] In this embodiment, the above-mentioned ablation detection model is used to predict the target frame of the collected ablation image, and 16 images are selected as a training batch.

[0062] In this embodiment, the sum of the training set bounding box loss and the training set classification loss during model training is selected as the training loss term, and compared with the current target detection algorithms yolov7 and spiking-yolo. Figure 3 As shown in the figure. Compared with yolov7 and spiking-yolo, this method has better performance during training. Compared with yolov7, this method does not improve much in training performance, but this method takes advantage of the variable multi-head self-attention to reduce the training time of this method.

[0063] The above specific embodiments are only several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A method for detecting lightning ablation of metal composite materials, characterized in that: The following steps are involved: Step S1: obtaining a lightning ablation image dataset of a metal composite material to be detected, preprocessing the image data, presetting a bounding box, and learning a two-dimensional representation of the metal composite material ablation image data through a predefined convolution layer, and then stretching it into one-dimensional data; Step S2: The data converted into a one-dimensional representation of the ablation image after feature extraction is sent to the Transformer encoder together with the position code, so that the network can further learn the relevant features. After deep neural network training, the obtained parameters are used by the decoder to learn the ablation phenomenon to be detected from the features; Step S3: Perform bipartite graph matching on the position and label of the feature learned in the previous step with the position and label of the actual data set, and calculate the loss function of the label and the loss function of the position based on the matching results; Step S4: using the learned model to predict the bounding box of the ablation image to be identified, determining whether ablation exists in the image and its location, and obtaining an image with an ablation range box; The step S1 specifically includes: Taking photos of metal composite materials with ablation phenomenon, developing an image dataset of metal composite materials with ablation phenomenon, extracting ablation feature maps of the original images through predefined convolutional layers, and setting a bounding box of ablation contained in the predicted image; A rotation-adaptive bounding box is used, and the position parameters of the rotation bounding box are defined as a six-tuple, ,in is the center point of the bounding box, is the width of the bounding box, is the height of the bounding box, is the rotation angle of the bounding box relative to the vertical line, Indicates whether there is ablation in the bounding box; Use k-means clustering method to select A preset bounding box size, which is used to determine the size of the bounding box during detection; The two-dimensional image after the convolution layer is stretched into one-dimensional data through the fully connected layer for the next step of processing; Using a variable multi-head self-attention mechanism, the calculation process of the multi-head self-attention layer is: the feature matrix and position encoding After adding, we get the query matrix , the bond matrix , value matrix That is the feature matrix , and the matrix containing self-attention information is calculated from these three matrices , The calculation formula of the matrix is: in, The query matrix The weight matrix obtained after a linear layer is is the aggregation matrix of multiple groups of weight matrices, is the position encoding matrix, Value Weight , From the query matrix The calculated position offset is passed through the variable multi-head self-attention layer to convert the residual block Add to the resulting matrix , and then normalize it, and then send the obtained matrix into the feedforward neural network, which consists of a fully connected layer, a Relu activation layer, a Dropout layer and a fully connected layer, and then pass the residual connection and normalization. This is the role of an encoding module. After passing through N encoding modules, the result is sent to the decoder.

2. A metal composite material lightning ablation detection method according to claim 1, characterized in that: The step S2 specifically includes: Stretch the two-dimensional data represented by the convolutional layer into one-dimensional data After that, the Transformer can meet the requirements of the data to be processed. At this time, the position code is extracted from it and the position code and data are combined. Input it into N encoding modules of the same composition to obtain a matrix that integrates all the attention head information, and then input it into the same N decoding modules of the same composition to obtain a The tensor is then fed into two feed-forward neural networks (FFNs) to determine whether there is ablation and the bounding box. is the training batch size.

3. A metal composite material lightning ablation detection method according to claim 2, characterized in that: Expanding trigonometric functions to dimensional vector as the representation of input information, different channels have different wavelengths; in, For the channel, is the total number of channels, is the location information, is a coordinate in the feature map, Calculated separately.

4. A metal composite material lightning ablation detection method according to claim 3, characterized in that: In the encoding process, N identical encoding modules are used, and the input of each encoding module is The image features and position encoding are outputted as feature matrices of the same dimension. Each encoding module consists of a multi-head self-attention layer and a feedforward neural network. is the height of the image feature, is the width of the image feature, is the training batch size.

5. A metal composite material lightning ablation detection method according to claim 2, characterized in that: In the decoding process, N identical modules are also used, where each module is composed of a variable multi-head self-attention layer MS1, a residual normalization layer, a variable multi-head self-attention layer MS2, a residual normalization layer, a feedforward neural network layer, and a residual normalization layer from bottom to top. The residual normalization layer is the same as the feedforward neural network layer and the residual normalization layer and feedforward neural network layer of the encoder, while the input of the multi-head self-attention layer is different from that of the encoder, which can be specifically expressed as follows: The input of MS1 is a position matrix with all zeros at the beginning and the category query matrix Add the query matrix to MS1 and key matrix , and the value matrix is ​​directly given by The result is obtained by normalizing the residual layer and then comparing it with the category query matrix Add as the query matrix of MS2 , and the value matrix of MS2 The output of the encoder Directly obtain the bond matrix of MS2 The output of the encoder With position encoding Add together to get, where the category query matrix It is also a learnable matrix, and the output of MS2 is used as the new position matrix After passing through a feedforward neural network layer and a residual normalization layer, it is used as the output of the entire decoder module.

6. A metal composite material lightning ablation detection method according to claim 1, characterized in that: The output of the decoder is sent to two different feedforward neural networks FFN1 and FFN2, where FFN1 is a category prediction network. This network can predict multiple categories, but only two categories need to be predicted according to needs, 0 for no ablation and 1 for ablation. FFN1 is composed of a fully connected layer, and FFN2 is a bounding box prediction network. FFN2 is composed of three fully connected layers. The final output is a data of size 6, and the center point, height, width, and angle of its bounding box are these five data.

7. A metal composite material lightning ablation detection method according to claim 1, characterized in that: The step S3 specifically includes: An end-to-end loss calculation method is used, that is, the difference between the categories and bounding boxes in the training set and the final predicted results is directly calculated, and the loss function is determined based on this. The binary matching algorithm is first used to select the bounding box corresponding to the label from the predicted bounding box, and the label loss of all bounding boxes is calculated in turn. and bounding box loss , and modify the various matrix parameters in the Transformer structure according to the loss; To calculate the loss function, we first need to use the bipartite graph matching algorithm to find the text box corresponding to the label, so that the final prediction result only calculates the loss with the matched true value. The formula for bipartite value matching is: in is a true value, is the predicted value, It is a bipartite graph matching algorithm. The purpose of this formula is to find a predicted value that minimizes the bipartite graph matching loss between it and the true value. After finding the bounding box corresponding to the label, the loss of this method consists of two parts, namely the classification loss With bounding box error loss , the total loss function is: The classification loss That is, whether it can be judged that the image has ablation phenomenon, and the bounding box error loss That is, the difference between the center point, height, width, and angle of the predicted bounding box and the real bounding box is calculated. The formula is expressed as: in, They are the center point coordinates, width, height, and angle of the bounding box respectively.

8. The method for detecting lightning ablation of metal composite materials according to claim 1, characterized in that: In the step S4, it specifically includes: After the training is completed, the image to be tested is preprocessed and input into the model. The model will automatically predict several bounding boxes and label probabilities. If the probability of the label exceeds the designed threshold, the result is directly output. If the label probability does not exceed, the bounding box is considered to have no label and the bounding box is not displayed.

Citation Information

Patent Citations

  • Special device and test method for ablation simulation test of fault current on pipeline

    CN107807088A

  • Single-stage rotating ship detection method based on full convolutional network

    CN114565824A