Classification method, device and equipment for hardware corrosion, medium and product

Abstract

The invention discloses a hardware fitting corrosion classification method, device and equipment, a medium and a product. The method comprises the following steps: acquiring a to-be-classified image corresponding to a target hardware fitting; inputting the to-be-classified image into a trained hardware fitting corrosion classification model to obtain features of a to-be-used hardware fitting; processing the to-be-processed channel features to obtain attention weight matrixes under different pooling layers; and determining a corrosion classification result corresponding to the target hardware fitting based on the attention weight matrixes, the to-be-channel-processed features and the to-be-sampled-processed features under different pooling layers. The problems of high cost and poor effect of hardware corrosion detection caused by manual inspection or laboratory detection in the prior art are solved, and the classification accuracy of hardware corrosion is improved, so that the safety and stability of a power grid transmission line are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a classification method, device, equipment, medium and product for metal fitting corrosion. Background Art

[0002] In recent years, with the growth of electricity demand, the requirements for stable operation of power systems have become increasingly higher. In power systems, once the hardware in the power transmission lines of the power system is corroded, it is likely to corrode and break, and even worse, it will cause the transmission line to trip and cause power transmission system failure. Therefore, in order to ensure the safe and stable operation of the power system, it is usually necessary to detect the corrosion of the hardware in the transmission line that protects, fixes, supports and maintains the stability of the transmission line.

[0003] At present, the main methods of hardware inspection are manual inspection and laboratory testing. Among them, manual inspection mainly relies on manual observation and manual recording of hardware corrosion. This method is subjective, resulting in poor detection results and high costs. Laboratory testing mainly involves taking hardware samples back to the laboratory and determining the corrosion level through chemical composition analysis. This not only has high detection costs, but also has the problem of detection lag, and cannot guarantee the safety of transmission line operation. Summary of the invention

[0004] The present invention provides a classification method, device, equipment, medium and product for hardware corrosion, so as to improve the accuracy of hardware corrosion classification, thereby ensuring the safety and stability of power grid transmission lines.

[0005] According to one aspect of the present invention, a method for classifying metal corrosion is provided, the method comprising:

[0006] Acquire the image to be classified corresponding to the target hardware;

[0007] The image to be classified is input into the trained hardware corrosion classification model to obtain the hardware features to be used; wherein the hardware features to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified by the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model;

[0008] Processing the channel-processing features to be processed to obtain attention weight matrices under different pooling layers;

[0009] Based on the attention weight matrices under different pooling layers, the features to be processed in the channels, and the features to be processed by sampling, the corrosion classification results corresponding to the target hardware are determined.

[0010] According to another aspect of the present invention, there is provided a device for classifying corrosion of hardware, the device comprising:

[0011] The module for acquiring images to be classified is used to acquire images to be classified corresponding to target hardware;

[0012] The hardware feature determination module to be used is used to input the image to be classified into the trained hardware corrosion classification model to obtain the hardware features to be used; wherein the hardware features to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified by the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model;

[0013] An attention weight matrix determination module is used to process the features to be processed by the channel to obtain attention weight matrices under different pooling layers;

[0014] The corrosion classification result determination module is used to determine the corrosion classification result corresponding to the target hardware based on the attention weight matrix under the different pooling layers, the channel processing features and the sampling processing features.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and a memory in communication with the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for classifying metal corrosion described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the classification method for metal fitting corrosion described in any embodiment of the present invention when executed.

[0019] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method for classifying corrosion of hardware as described in any embodiment of the present invention.

[0020] The technical solution of the embodiment of the present invention obtains an image to be classified corresponding to the target hardware; inputs the image to be classified into a trained hardware corrosion classification model to obtain the features of the hardware to be used; wherein the features of the hardware to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified based on the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled and processed based on the second convolution module in the hardware corrosion classification model; the features to be processed through channels are processed to obtain the attention weight matrices under different pooling layers; based on the attention weight matrices under different pooling layers, the features to be processed through channels and the features to be sampled, The rust classification results corresponding to the target hardware solve the problem of high cost and poor effect of hardware rust detection caused by manual inspection or laboratory testing in the existing technology. It realizes that the first convolution module in the trained hardware rust classification model processes the classified image to obtain the features to be sampled, and the second convolution module processes the features to be sampled to obtain the features to be channel processed, thereby extracting image features from different dimensions. Furthermore, by processing the features to be channel processed, the attention weight matrices under different pooling layers are obtained, so that the attention weight matrices under different pooling layers are used to enhance the feature information of hardware and improve the accuracy of hardware rust classification, thereby effectively ensuring the safety and stability of power grid transmission lines.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flow chart of a method for classifying metal corrosion provided in accordance with the first embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of a classification method for metal fitting corrosion according to Embodiment 1 of the present invention;

[0025] Figure 3 is a schematic diagram for representing the rust classification result after the original image is classified according to the first embodiment of the present invention;

[0026] Figure 4This is a flow chart of a method for classifying metal corrosion provided in accordance with Embodiment 2 of the present invention;

[0027] Figure 5 A schematic diagram of a classification method for metal fitting corrosion according to a second embodiment of the present invention;

[0028] Figure 6 A flow chart of a method for classifying metal corrosion provided according to Embodiment 3 of the present invention;

[0029] Figure 7 A flow chart of a method for classifying metal corrosion provided according to a fourth embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the structure of a classification device for metal fitting corrosion provided in accordance with Embodiment 5 of the present invention;

[0031] Fig. 9 It is a schematic diagram of the structure of an electronic device for implementing the classification method for metal fitting corrosion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] Figure 1This is a flow chart of a method for classifying metal corrosion according to the first embodiment of the present invention. This embodiment is applicable to the case of identifying the degree of metal corrosion. The method can be performed by a device for classifying metal corrosion. The device for classifying metal corrosion can be implemented in the form of hardware and / or software. The device for classifying metal corrosion can be configured in a computing device. Figure 1 As shown, the method includes:

[0036] S110, obtaining an image to be classified corresponding to the target hardware.

[0037] Among them, the target hardware may refer to the hardware whose corrosion level is desired to be identified, such as the hardware may be a metal component used to connect, fix or support power equipment in a transmission line. For example, the hardware may include but is not limited to suspension clamps, tension clamps, connection hardware (such as insulators), connection hardware (such as crimping tubes, repair tubes, etc.), protection hardware (such as anti-vibration hammers, wire protection lines, etc.), wire pulling hardware, jumper hardware, etc. In practical applications, the image to be classified may be collected based on a camera device, and the number of hardware in the entry image may be one or more. If the number of hardware is one, the hardware may be used as the target hardware; if the number of hardware is more than one, all or part of the hardware may be used as the target hardware. In other words, the number of target hardware included in the image to be classified may be one or more.

[0038] In this embodiment, when it is necessary to determine whether the hardware in the transmission line is corroded, an image including the hardware can be collected, and the image can be used as the image to be classified; alternatively, an image of the hardware in the transmission line can be collected based on a camera device, and an image containing the hardware can be extracted from the image as the image to be classified, so as to determine the degree of corrosion of the hardware by analyzing the image to be classified.

[0039] S120, inputting the image to be classified into the trained hardware corrosion classification model to obtain the features of the hardware to be used; the features of the hardware to be used include features to be sampled and processed and features to be channel processed.

[0040] Among them, the hardware corrosion classification model is used to classify the hardware corrosion conditions. The features to be sampled are obtained by processing the images to be classified by the first convolution module in the hardware corrosion classification model. The features to be channel processed are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model.

[0041] In this embodiment, the image to be classified is processed by the first convolution module in the hardware corrosion classification model to obtain the features to be sampled. Then, the features to be sampled are input into the second convolution module in the hardware corrosion classification model to obtain the features to be processed. The first convolution module and the second convolution module have different model structures and are used to extract hardware features from different dimensions.

[0042] Optionally, the first convolution module includes a first convolution unit, a first downsampling unit, a second convolution unit, a second downsampling unit, a third convolution unit, a third downsampling unit, and a fourth convolution unit; the second convolution module includes a fourth downsampling unit and a fifth convolution unit. Each downsampling unit may be a maximum pooling layer (MaxPooling Layer). The maximum pooling layer is used to extract the maximum value in the input local area as the output, which can effectively retain the strong features in the image such as edges, textures, etc., so that these strong feature information can be better retained in subsequent layers. The convolution unit is used to perform convolution processing on the input data. Convolution processing (Convolution Operation) refers to sliding a convolution kernel on the input data and performing weighted summation on the local area in the input data to extract the local features of the input data. The number of convolution kernels, window size and padding parameters of each convolution unit can be seen in Table 1 below.

[0043] Table 1

[0044] Number of convolution kernels Window size Padding The first convolution unit 64 3×3 1 The second convolution unit 128 3×3 1 The third convolution unit 256 3×3 1 The fourth convolution unit 512 3×3 1 Fifth convolution unit 512 3×3 1

[0045] Among them, the convolution kernel refers to the weighted average of pixels in a small area of ​​the input image to become each corresponding pixel in the output image when a given input image is processed, where the weight is defined by a function called the convolution kernel. The window size refers to the size of the convolution kernel. Padding refers to padding the edges of the input during the convolution operation to control the size and shape of the output feature map.

[0046] On the basis of the above technical solution, the image to be classified is input into the trained hardware corrosion classification model to obtain the hardware features to be used, including: processing the image to be classified based on the first convolution unit in the first convolution module to obtain the first feature to be downsampled; processing the first feature to be downsampled based on the first downsampling unit in the first convolution module to obtain the first feature to be processed; processing the first feature to be processed based on the second convolution unit in the first convolution module to obtain the second feature to be downsampled; processing the second feature to be downsampled based on the second downsampling unit in the first convolution module to obtain the second feature to be processed; processing the second feature to be processed based on the third convolution unit in the first convolution module Processing is performed to obtain a third feature to be downsampled; based on the third downsampling unit in the first convolution module, the third feature to be downsampled is processed to obtain a third feature to be processed; based on the fourth convolution unit in the first convolution module, the third feature to be processed is processed to obtain a fourth feature to be downsampled; based on the fourth downsampling unit in the second convolution module, the fourth feature to be downsampled is processed to obtain a fourth feature to be processed; based on the fifth convolution unit in the second convolution module, the fourth feature to be processed is processed to obtain a feature to be channel processed, and based on at least one of the first feature to be downsampled, the second feature to be downsampled, the third feature to be downsampled and the fourth feature to be downsampled, the feature to be sampled and processed is determined.

[0047] In this embodiment, see Figure 2First, the image to be classified can be used as the input of the first convolution unit in the first convolution module, and after convolution processing, the first feature to be downsampled can be obtained. The first feature to be downsampled can be used as the input of the first downsampling unit (MaxPooling Layer), and after maximum pooling processing, the first feature to be processed can be obtained. The first feature to be processed can be used as the input of the second convolution unit, and after convolution processing, the second feature to be downsampled can be obtained. The second feature to be downsampled can be used as the input of the second downsampling unit, and after maximum pooling processing, the second feature to be processed can be obtained. The second feature to be processed can be used as the input of the third convolution unit, and after convolution processing, the third feature to be downsampled can be obtained. The third feature to be downsampled can be used as the input of the third downsampling unit, and after maximum pooling processing, the third feature to be processed can be obtained. The third feature to be processed can be used as the input of the fourth convolution unit, and after convolution processing, the fourth feature to be downsampled can be obtained. The fourth feature to be processed can be used as the input of the fourth downsampling unit, and after maximum pooling processing, the fourth feature to be processed can be obtained. The fourth feature to be processed is used as the input of the fifth convolution unit, and convolution processing is performed to obtain the feature to be processed. Furthermore, the first feature to be downsampled, the second feature to be downsampled, the third feature to be downsampled, and the fourth feature to be downsampled can all be used as features to be sampled and processed. The technical solution provided in this embodiment better retains the strong feature information in the image through the maximum pooling layer, and through the convolution layer, it can effectively capture the local features of the image and reduce the computational complexity.

[0048] The technical solution provided in this embodiment uses a convolutional layer to extract local corrosion patterns or texture information in an image; secondly, a nonlinear activation function is used to enhance the expressive power of the model so that it can capture more complex corrosion features; finally, a maximum pooling layer is used to enhance strong features in the image, and the size of the feature map is reduced by downsampling, thereby reducing the amount of calculation and retaining key information. The whole process is executed four times in a loop, and a richer feature representation is gradually extracted to improve classification accuracy. It should be noted that the size and shape of the convolution kernel and the stride of the convolution can also be dynamically adjusted according to different actual needs. By changing the weight of the convolution kernel, different feature extraction and filtering effects can be achieved.

[0049] S130, processing the channel processing features to be processed to obtain the attention weight matrix under different pooling layers.

[0050] The pooling layer includes but is not limited to the maximum pooling layer, the average pooling layer, the mixed pooling, the Lp pooling, the sophisticated pooling and the spatial pyramid pooling (SPP), etc. The attention weight matrix may include the weight of each pixel feature in the feature to be processed by the channel.

[0051] In this embodiment, different pooling layers can be used to divide the features to be processed by the channels into non-overlapping areas, and perform a convergence operation on each area to extract feature data under different feature dimensions in the image, and analyze the importance of each feature in the feature information extracted by each pooling layer, respectively, to obtain the attention weight matrix under different pooling layers. The attention weight matrix under different pooling layers can express the weight information of the features extracted by different pooling layers.

[0052] S140. Determine the corrosion classification result corresponding to the target hardware based on the attention weight matrix under different pooling layers, the features to be processed in the channel, and the features to be processed in the sampling.

[0053] In this embodiment, the attention weight matrix under different pooling layers can be used to weight the channel processing features to enhance their feature representation. Then, the weighted channel processing features and the sample processing features can be combined to determine the final hardware feature map, and the hardware feature map can be analyzed to obtain the rust classification result corresponding to the target hardware. It should be noted that the rust classification result can include one or more, and different display information (such as color) can be used to represent different rust levels. For example, see Figure 3 , the image on the left represents the original image (i.e. the image to be classified), and the image on the right represents the classification map output by the model (the classification map includes different corrosion classification results). For example, green represents light corrosion, red represents moderate corrosion, and yellow represents severe corrosion. Through clear color labeling, users can quickly and intuitively understand the specific corrosion conditions of hardware in the transmission line. Since different corrosion levels have different priorities in the decision-making of hardware maintenance, it is possible to accurately distinguish different corrosion levels of hardware such as light, medium, and severe, so as to facilitate targeted maintenance plans, optimize resource allocation, and reduce operation and maintenance costs.

[0054] The technical solution provided in this embodiment obtains the image to be classified corresponding to the target hardware; inputs the image to be classified into the trained hardware corrosion classification model to obtain the features of the hardware to be used; wherein the features of the hardware to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified based on the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled and processed based on the second convolution module in the hardware corrosion classification model; the features to be processed through channels are processed to obtain the attention weight matrices under different pooling layers; based on the attention weight matrices under different pooling layers, the features to be processed through channels and the features to be sampled, The rust classification results corresponding to the target hardware solve the problem of high cost and poor effect of hardware rust detection caused by manual inspection or laboratory testing in the prior art. It realizes that the first convolution module in the trained hardware rust classification model processes the classified image to obtain the features to be sampled, and the second convolution module processes the features to be sampled to obtain the features to be channel processed, thereby extracting image features from different dimensions. Furthermore, by processing the features to be channel processed, the attention weight matrices under different pooling layers are obtained, so that the attention weight matrices under different pooling layers are used to enhance the feature information of hardware and improve the accuracy of hardware rust classification, thereby effectively ensuring the safety and stability of power grid transmission lines.

[0055] Embodiment 2

[0056] Figure 4 It is a flow chart of a classification method for metal corrosion provided according to the second embodiment of the present invention. On the basis of the above embodiment, the metal corrosion classification model also includes a channel attention module, and the channel attention module includes a maximum pooling layer and an average pooling layer; accordingly, the "S130 step" can be further refined. Its specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above embodiment are not repeated here.

[0057] like Figure 4 As shown, the method specifically comprises the following steps:

[0058] S210: Acquire an image to be classified corresponding to the target hardware.

[0059] S220, input the image to be classified into the trained hardware corrosion classification model to obtain the features of the hardware to be used; the features of the hardware to be used include features to be sampled and processed and features to be channel processed.

[0060] S230, processing the channel processing features based on the maximum pooling layer in the channel attention module to obtain the first channel features.

[0061] Among them, the channel attention module is used to adaptively adjust the weights of different channels so that the model can pay more attention to the feature information that is most important for the classification of rust grades.

[0062] In this embodiment, a maximum pooling layer can be used to extract the maximum value in the local area of ​​the channel processing feature as the output, retaining the significant features in the image, such as edges, textures, etc., to obtain the first channel feature.

[0063] S240, processing the channel processing features based on the average pooling layer in the channel attention module to obtain a second channel feature.

[0064] In this embodiment, an average pooling layer can be used to calculate the average value of the local area in the channel processing feature, and the average value is used as the output to obtain the second channel feature, so that the overall feature information in the image can be retained in the second channel feature.

[0065] For example, see Figure 2 , add the channel attention module (CPCA) after the fifth convolutional unit. Figure 5 As shown in the figure, the obtained channel processing features (size is H×W×C, where H is the height, W is the width, and C is the number of channels) are respectively passed through the average pooling layer (Average Pooling Layer) and the maximum pooling layer (Max Pooling Layer). The function of these two layers is to compress the global information of the feature map and generate two feature vectors of size 1×1×C, which represent the channel features under different pooling layers, namely the second channel features and the first channel features.

[0066] S250. Based on the first channel features, determine the attention weight matrix under the maximum pooling layer, and based on the second channel features, determine the attention weight matrix under the average pooling layer.

[0067] In this embodiment, the importance of each feature can be analyzed based on the strong feature attributes and position information in the first channel feature, and the attention weight matrix under the maximum pooling layer can be determined, and the attention weight matrix can express the weight information of the strong features in the image. The importance of each feature can also be analyzed based on the average feature attributes and position information in the second channel feature, and the attention weight matrix under the average pooling layer can be determined, and the attention weight matrix can express the weight information of the average feature in the image.

[0068] Optionally, the channel attention module also includes a multi-layer perceptron and an activation function. Accordingly, based on the first channel feature, the attention weight matrix under the maximum pooling layer is determined, and based on the second channel feature, the attention weight matrix under the average pooling layer is determined, including: inputting the first channel feature and the second channel feature into the multi-layer perceptron in the channel attention module respectively, obtaining a third channel feature corresponding to the first channel feature and a fourth channel feature corresponding to the second channel feature; processing the third channel feature according to the activation function in the channel attention module to obtain the attention weight matrix under the maximum pooling layer, and processing the fourth channel feature according to the activation function to obtain the attention weight matrix under the average pooling layer.

[0069] The activation function may be ReLU (Rectified Linear Unit). The expression of the activation function may be: ReLU = max(0, x), which means that when the input value x is greater than zero, the activation function directly outputs the input value; when the input value x is less than or equal to zero, the output of the activation function is zero.

[0070] Continue to see Figure 5 , the two 1×1×C channel feature vectors (i.e., the first channel feature and the second channel feature) are respectively sent to two layers of multilayer perceptrons (MLPs) in sequence. In the first layer, the number of channels is reduced to C / r (r is the reduction rate, for example, r is 4), and in the second layer, the number of channels is restored to C. After being processed by the multilayer perceptron, the output feature vectors (i.e., the third channel feature and the fourth channel feature) are respectively passed through the activation function so that their values ​​are limited between 0 and 1, thereby forming the final weight distribution, and obtaining the attention weight matrix under the maximum pooling layer and the attention weight matrix under the average pooling layer. The technical solution provided in this embodiment calculates the weight of each channel by inputting the first channel feature and the second channel feature into the multilayer perceptron (MLP), and then multiplies these weights with the corresponding channel features, thereby enhancing those key channel information, so that it can help the model more accurately capture the significant features related to the corrosion of metal fittings and improve the classification accuracy.

[0071] S260, based on the attention weight matrices under different pooling layers, the features to be processed in the channels, and the features to be processed by sampling, determine the rust classification results corresponding to the target hardware.

[0072] The technical solution provided in this embodiment uses a channel attention module as an attention mechanism to perform global maximum pooling and global average pooling operations on the features of each channel to calculate the weights of each channel. Subsequently, these weights are used to weight the response of the channel features, thereby enhancing the sensitivity of the model to important channel features and improving the classification accuracy of the model for different degrees of corrosion.

[0073] Embodiment 3

[0074] Figure 6 According to the flowchart of a classification method for metal fitting corrosion provided in the third embodiment of the present invention, on the basis of the above embodiment, the "step S140" can be further refined. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above embodiment are not repeated here.

[0075] like Figure 6 As shown, the method specifically comprises the following steps:

[0076] S310: Acquire an image to be classified corresponding to the target hardware.

[0077] S320, input the image to be classified into the trained hardware corrosion classification model to obtain the features of the hardware to be used; the features of the hardware to be used include features to be sampled and processed and features to be channel processed.

[0078] S330, processing the channel processing features to be processed to obtain the attention weight matrix under different pooling layers.

[0079] S340: Determine the first feature to be upsampled based on the attention weight matrices under different pooling layers and the features to be processed by the channels.

[0080] Continue to see Figure 5 , the attention weight matrices under different pooling layers can be added to obtain the added attention weight matrix, and the added attention weight matrix and the feature to be processed through the channel can be multiplied to obtain the first feature to be upsampled. Alternatively, it can also be: the feature to be processed through the channel can be multiplied with the attention weight matrices under different pooling layers respectively to obtain multiple multiplied features to be processed through the channel, and then the multiple multiplied features to be processed through the channel can be fused to obtain the first feature to be upsampled.

[0081] S350: Determine a corrosion classification result corresponding to the target hardware based on the first feature to be sampled and the feature to be sampled and processed.

[0082] In this embodiment, the first feature to be sampled and the feature to be sampled and processed may be analyzed based on the hardware corrosion classification model to obtain a corrosion classification result corresponding to the target hardware.

[0083] Optionally, the hardware corrosion classification model further includes a first upsampling unit, a sixth convolution unit, a second upsampling unit, a seventh convolution unit, a third upsampling unit, an eighth convolution unit, a fourth upsampling unit and a ninth convolution unit. The sampling method adopted by the upsampling unit may include but is not limited to bilinear interpolation, cubic spline interpolation, deconvolution, depooling, and the like.

[0084] Correspondingly, based on the first feature to be upsampled and the feature to be sampled and processed, the corrosion classification result corresponding to the target hardware is determined, including: processing the first feature to be upsampled based on the first upsampling unit in the hardware corrosion classification model to obtain the first feature to be spliced; based on the fourth feature to be downsampled and the first feature to be spliced ​​in the feature to be sampled and processed, obtaining the fifth feature to be processed; based on the sixth convolution unit in the hardware corrosion classification model, processing the fifth feature to be processed to obtain the second feature to be upsampled; based on the second upsampling unit in the hardware corrosion classification model, processing the second feature to be upsampled to obtain the second feature to be spliced; based on the third feature to be downsampled and the second feature to be spliced ​​in the feature to be sampled and processed, obtaining the sixth feature to be processed; based on the seventh convolution unit in the hardware corrosion classification model, processing the sixth feature to be processed The third feature to be up-sampled is processed based on the third upsampling unit in the hardware corrosion classification model to obtain the third feature to be spliced; the second feature to be down-sampled and the third feature to be spliced ​​in the feature to be sampled are processed to obtain the seventh feature to be processed; the seventh feature to be processed is processed based on the eighth convolution unit in the hardware corrosion classification model to obtain the fourth feature to be up-sampled; the fourth feature to be up-sampled is processed based on the fourth upsampling unit in the hardware corrosion classification model to obtain the fourth feature to be spliced; the eighth feature to be processed is processed based on the first feature to be down-sampled and the fourth feature to be spliced ​​in the feature to be sampled; the eighth feature to be processed is processed based on the ninth convolution unit in the hardware corrosion classification model to determine the corrosion classification result corresponding to the target hardware.

[0085] Continue to see Figure 2, the first feature to be upsampled can be input into the first upsampling unit (up Sampling), the feature is restored, and the first feature to be spliced ​​is obtained. Then, the fourth feature to be downsampled in the feature to be sampled and processed is spliced ​​with the first feature to be spliced ​​to obtain the fifth feature to be processed. The fifth feature to be processed is input into the sixth convolution unit, and convolution processing is performed to obtain the second feature to be upsampled. The second feature to be upsampled is input into the second upsampling unit, and the feature is restored to obtain the second feature to be spliced. Then, the third feature to be downsampled in the feature to be sampled and processed and the second feature to be spliced ​​are spliced ​​to obtain the sixth feature to be processed. The sixth feature to be processed is input into the seventh convolution unit for convolution processing to obtain the third feature to be upsampled. The third feature to be upsampled is input into the third upsampling unit to obtain the third feature to be spliced. Then, the second feature to be downsampled in the feature to be sampled and processed and the third feature to be spliced ​​are spliced ​​to obtain the seventh feature to be processed. The seventh feature to be processed is input into the eighth convolution unit for convolution processing to obtain the fourth feature to be upsampled. The fourth feature to be upsampled is input into the fourth upsampling unit to obtain the fourth feature to be spliced. Then, the first feature to be downsampled and the fourth feature to be spliced ​​in the sampled processing feature are spliced ​​to obtain the eighth feature to be processed. The eighth feature to be processed is input into the ninth convolution unit to perform convolution processing on the eighth feature to be processed, and then the eighth feature to be processed after the convolution processing is processed by the fully convolutional network, and a prediction map of the corrosion grade of the target hardware is output, which includes at least one corrosion classification result.

[0086] The advantage of this setting is that the original information of the image is retained by means of skip connections, which helps the model to restore the original detail features after multiple upsamplings, and finally outputs the corrosion classification results of the target hardware through the fully convolutional network, ensuring the accuracy of the corrosion classification.

[0087] The technical solution provided in this embodiment enhances the channel processing features based on the attention weight matrices under different pooling layers to obtain the first features to be upsampled, and then determines the corrosion classification results corresponding to the target hardware through the first features to be upsampled and the features to be sampled, thereby improving the classification accuracy of the model.

[0088] Embodiment 4

[0089] Figure 7According to the flowchart of a metal fitting rust classification method provided in the fourth embodiment of the present invention, on the basis of the above-mentioned embodiment, before the image to be classified is input into the trained metal fitting rust classification model to obtain the features of the metal fittings to be used, the metal fitting rust classification model can be pre-trained. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0090] like Figure 7 As shown, the method specifically comprises the following steps:

[0091] S410, obtaining a plurality of training samples, wherein the training samples include a hardware corrosion image and a graded corrosion annotation map corresponding to the hardware corrosion image; the graded corrosion annotation map includes at least one annotated corrosion grade.

[0092] In order to improve the accuracy of the model corrosion classification, as many and rich metal corrosion images as possible can be obtained, and the metal corrosion images can be annotated with corrosion grades to obtain the corresponding graded corrosion annotation images. Thus, a training sample set can be obtained.

[0093] In this embodiment, drones can be used to collect rich and high-quality images of metal corrosion in power transmission lines to obtain metal corrosion images. Annotation tools are used to annotate the metal corrosion images with corrosion levels according to different degrees of corrosion. Specifically, the annotation process may include: installing a programming environment, configuring annotation tools, and annotating corrosion levels.

[0094] Among them, the method of annotating the rust level can be: after completing the environment configuration, open the annotation tool and load the hardware corrosion image to be annotated. For each hardware corrosion image, accurate polygon annotation is performed according to the different degrees of its corrosion. When annotating, the selection should be made on the basis of ensuring the balance of categories as much as possible to ensure that the distribution of images of different corrosion levels is reasonable and uniform. During the annotation process, the boundaries of different corrosion areas can be accurately marked by drawing polygonal contours from the hardware corrosion image to obtain an annotation file. After the annotation is completed, the annotation file can also be converted into a label image in a preset format. In this embodiment, a random array can be used to randomly divide the created hardware corrosion data set into a training sample set and a test sample set in a ratio of 7:3. The initial hardware corrosion classification model is trained using the training sample set, and the trained hardware corrosion classification model is verified using the test sample set to ensure the accuracy and robustness of the model corrosion classification.

[0095] S420: For each training sample, the hardware corrosion image in the training sample is input into the hardware corrosion classification model to be trained to obtain a predicted corrosion classification map.

[0096] In this embodiment, a hardware corrosion classification model to be trained can be pre-constructed. The hardware corrosion classification model has the same model structure as the above-trained hardware corrosion classification model. After the construction is completed, the model is trained using a pre-divided training sample set. The hardware corrosion images in the training samples are input into the hardware corrosion classification model to be trained, and the model predicts the corrosion level to obtain a predicted corrosion classification map. The prediction method is the same as the method of inputting the image to be classified into the trained hardware corrosion classification model to determine the corresponding corrosion classification result of the target hardware, which will not be repeated here.

[0097] S430, determining a prediction result corresponding to the annotated rust grade in the graded rust annotated map in the training sample from the predicted rust classification map.

[0098] In this embodiment, the predicted probability of the annotated rust grade in the graded rust annotation map in the training sample can be found from the predicted rust classification map as the prediction result. For example, the graded rust annotation map includes the annotation result of the heavy rust grade. At this time, if the predicted probability of the heavy rust grade in the predicted rust classification map is 70%, then the predicted probability of the heavy rust grade 70% is extracted as the prediction result.

[0099] S440: Determine a loss value based on the prediction result and the loss function in the hardware corrosion classification model.

[0100] In this embodiment, the prediction result can be used as an input parameter of the loss function in the hardware corrosion classification model to obtain a loss value. The loss function can be a Focal Loss loss function, and the formula of the loss function can be expressed as: FL(p t )=-(1-p t ) γ log(p t ). Among them, p t It represents the predicted probability of the model for the true category of the sample (i.e., the predicted result), γ is an adjustable hyperparameter, FL(p t ) represents the loss value.

[0101] The technical solution provided in this embodiment can effectively solve the problem of uneven distribution of the number of samples with different corrosion levels by adopting the Focal Loss loss function to optimize the parameter update of the model. Compared with the cross entropy loss function, the Focal Loss loss function adjusts the weights so that the model can still pay more attention to difficult-to-classify samples when facing a large number of easy-to-classify samples, thereby improving the learning effect of the model. By introducing an adjustable hyperparameter as a decay factor, the parameter controls the loss decrease rate of easy-to-classify samples, thereby reducing the weight of easy-to-classify samples to reduce their contribution to the overall loss. This makes the model more focused on samples with higher uncertainty or difficult to classify, thereby improving the model's learning ability for these complex samples and improving the overall performance and robustness of the model.

[0102] S450: Modify the model parameters in the hardware corrosion classification model based on the loss value to obtain a pre-trained hardware corrosion classification model.

[0103] In this embodiment, the model parameters in the hardware corrosion classification model can be corrected by the loss value. During the model training process, the loss value gradually decreases as the prediction result approaches the true value until the model converges to the optimal state, until the pre-trained hardware corrosion classification model is obtained. The technical solution provided in this embodiment can use the Adam optimizer (Adaptive Moment Estimation) to optimize the parameters, and the initial learning rate is set between 0.005 and 1e-6, and the learning rate of each parameter is adaptively adjusted to improve the convergence speed and stability of the model.

[0104] Specifically, the Adam optimizer calculates the first-order moment (i.e., the mean of the gradient) and the second-order moment (i.e., the mean of the square of the gradient) of the current gradient in each iteration, and uses these moments to update the model parameters. By adaptively adjusting the learning rate of each parameter, it can achieve better convergence performance under the gradient conditions of different parameters, effectively avoid falling into the problem of local optimal solution or premature convergence, and ensure that the model has strong generalization ability in complex hardware corrosion grade classification tasks.

[0105] In order to ensure the classification accuracy and robustness of the model, after obtaining the pre-trained hardware corrosion classification model, the pre-trained hardware corrosion classification model can also be verified using test samples. If the verification passes, the image to be classified can be input into the trained hardware corrosion classification model to obtain the features of the hardware to be used.

[0106] In this embodiment, the method for verifying the pre-trained hardware corrosion classification model can be: obtaining multiple test samples, the test samples include hardware corrosion images and corrosion grade annotation maps corresponding to the hardware corrosion images; the corrosion grade annotation map includes multiple first pixel points and annotated corrosion grades corresponding to the first pixel points; for each test sample, the hardware corrosion image in the test sample is input into the pre-trained hardware corrosion classification model to obtain a test corrosion classification map; the test corrosion classification map includes multiple second pixel points and corrosion prediction results corresponding to the second pixel points; based on the multiple first pixel points in the corrosion grade annotation map in the test sample and the annotated corrosion grades corresponding to the first pixel points, and the multiple second pixel points in the corresponding test corrosion classification map and the corrosion prediction results corresponding to the second pixel points, determine an indicator value corresponding to at least one preset evaluation indicator; when the indicator value meets the preset conditions, obtain the trained hardware corrosion classification model.

[0107] In this embodiment, the test sample set can be used to evaluate the accuracy of the pre-trained hardware corrosion classification model, and the evaluation targets include three corrosion levels: light corrosion, moderate corrosion, and severe corrosion. In order to comprehensively measure the classification performance of the model, a number of preset evaluation indicators are used for evaluation. These preset evaluation indicators include, but are not limited to: Overall Accuracy (OA) indicator, Mean Intersection over Union (MIoU) indicator, and Mean Pixel Accuracy (MPA) indicator. Among them, the overall accuracy indicator is used to evaluate the proportion of samples correctly classified by the model in the entire test sample set, reflecting the overall classification accuracy of the model. The average category intersection over Union indicator is used to measure the classification performance of the model at each corrosion level. The average pixel accuracy indicator is used to reflect the accuracy of the model's classification at the pixel level, and is a statistical measurement indicator for whether the classification of each pixel point is correct.

[0108] Specifically, the hardware corrosion image in the test sample can be input into the pre-trained hardware corrosion classification model to obtain a test corrosion classification map. According to the calculation formula for determining the index value corresponding to each preset evaluation index, the index value corresponding to at least one preset evaluation index is determined in combination with the test corrosion classification map and the corrosion grade annotation map. When the index value meets the preset conditions, it can be considered that the hardware corrosion classification model is trained.

[0109] For example, the calculation formula for determining the indicator value corresponding to the overall accuracy indicator is: Among them, C(i,j) indicates that the test rust classification map corresponding to the training sample is consistent with the rust grade annotation map; Indicates the total number of training samples that are consistent with the test rust classification map and the rust grade annotation map; N i Indicates the total number of training samples. This indicator value represents the number of correctly classified pixels of each type divided by the actual number of pixels of each type.

[0110] The calculation formula for determining the index value corresponding to the average category intersection ratio index is: Among them, p ii Indicates the pixels that are consistent between the test rust classification map and the rust grade annotation map, p ji and p ij Indicates the pixels where there are inconsistencies between the test rust classification map and the rust grade annotation map.

[0111] The calculation formula for determining the index value corresponding to the average pixel accuracy index is:

[0112] This indicator value represents the average accuracy of each type of pixel.

[0113] The technical solution provided in this embodiment collects a large number of corrosion images of power transmission line hardware, and finely annotates these images according to different degrees of corrosion, to construct a test sample set and a test sample set. Subsequently, by incorporating a channel attention module into the model, the feature information of the image is extracted through a convolutional network, and an attention mechanism is introduced to enhance the model's ability to capture key corrosion features, especially in complex or uneven corrosion grade samples. It shows stronger focusing ability and accuracy, so that it can better adapt to complex corrosion conditions and improve classification performance. At the same time, these preset evaluation indicators are used to measure the classification performance of the model at the global and local levels, so that it can effectively reflect the model's ability to distinguish between different corrosion levels and ensure the accuracy and robustness of the model.

[0114] Embodiment 5

[0115] Figure 8 1 is a schematic diagram of the structure of a classification device for metal fitting corrosion provided in Embodiment 5 of the present invention. Figure 8 As shown, the device includes: an image acquisition module 510 for classification, a hardware feature determination module 520 for use, an attention weight matrix determination module 530 and a corrosion classification result determination module 540.

[0116] Among them, the image acquisition module 510 to be classified is used to obtain the image to be classified corresponding to the target hardware; the hardware feature determination module 520 to be used is used to input the image to be classified into the trained hardware corrosion classification model to obtain the hardware features to be used; wherein the hardware features to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified by the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model; the attention weight matrix determination module 530 is used to process the features to be processed through channels to obtain the attention weight matrices under different pooling layers; the corrosion classification result determination module 540 is used to determine the corrosion classification result corresponding to the target hardware based on the attention weight matrices under different pooling layers, the features to be processed through channels and the features to be sampled.

[0117] The technical solution of this embodiment is to obtain an image to be classified corresponding to the target hardware; input the image to be classified into a trained hardware corrosion classification model to obtain the features of the hardware to be used; wherein the features of the hardware to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified based on the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled and processed based on the second convolution module in the hardware corrosion classification model; the features to be processed through channels are processed to obtain the attention weight matrices under different pooling layers; based on the attention weight matrices under different pooling layers, the features to be processed through channels and the features to be sampled, the hardware features to be classified are determined. The rust classification results corresponding to the marked hardware solve the problem of high cost and poor effect of hardware rust detection caused by manual inspection or laboratory testing in the prior art. It realizes that the first convolution module in the trained hardware rust classification model processes the classified image to obtain the features to be sampled, and the second convolution module processes the features to be sampled to obtain the features to be channel processed, thereby extracting image features from different dimensions. Furthermore, by processing the features to be channel processed, the attention weight matrices under different pooling layers are obtained, so that the attention weight matrices under different pooling layers are used to enhance the feature information of hardware and improve the accuracy of hardware rust classification, thereby effectively ensuring the safety and stability of power grid transmission lines.

[0118] On the basis of the above device, optionally, the first convolution module includes a first convolution unit, a first downsampling unit, a second convolution unit, a second downsampling unit, a third convolution unit, a third downsampling unit, and a fourth convolution unit; the hardware feature determination module 520 to be used includes:

[0119] A first feature to be downsampled determining unit, configured to process the image to be classified based on the first convolution unit in the first convolution module to obtain a first feature to be downsampled;

[0120] a first feature-to-be-processed determining unit, configured to process the first feature-to-be-downsampled based on the first downsampling unit in the first convolution module to obtain a first feature-to-be-processed;

[0121] A second feature to be downsampled determining unit, configured to process the first feature to be processed based on the second convolution unit in the first convolution module to obtain a second feature to be downsampled;

[0122] a second feature-to-be-processed determining unit, configured to process the second feature-to-be-downsampled based on the second downsampling unit in the first convolution module to obtain a second feature-to-be-processed;

[0123] A third feature to be downsampled determining unit, configured to process the second feature to be processed based on the third convolution unit in the first convolution module to obtain a third feature to be downsampled;

[0124] a third feature-to-be-processed determining unit, configured to process the third feature-to-be-downsampled based on the third downsampling unit in the first convolution module to obtain a third feature-to-be-processed;

[0125] a fourth feature to be downsampled determining unit, configured to process the third feature to be processed based on the fourth convolution unit in the first convolution module to obtain a fourth feature to be downsampled;

[0126] a fourth feature-to-be-processed determining unit, configured to process the fourth feature-to-be-downsampled based on the fourth downsampling unit in the second convolution module to obtain a fourth feature-to-be-processed;

[0127] The hardware feature determination unit to be used is used to process the fourth feature to be processed based on the fifth convolution unit in the second convolution module to obtain the feature to be channel processed, and determine the feature to be sampled and processed based on at least one of the first feature to be downsampled, the second feature to be downsampled, the third feature to be downsampled and the fourth feature to be downsampled.

[0128] On the basis of the above device, optionally, the hardware corrosion classification model also includes a channel attention module, and the channel attention module includes a maximum pooling layer and an average pooling layer; the attention weight matrix determination module 530 includes:

[0129] A first channel feature determination unit is used to process the to-be-processed channel feature based on the maximum pooling layer in the channel attention module to obtain a first channel feature;

[0130] A second channel feature determination unit is used to process the channel feature to be processed based on the average pooling layer in the channel attention module to obtain a second channel feature;

[0131] An attention weight matrix determination unit is used to determine the attention weight matrix under the maximum pooling layer based on the first channel features, and to determine the attention weight matrix under the average pooling layer based on the second channel features.

[0132] Based on the above device, optionally, the channel attention module further includes a multilayer perceptron and an activation function; the attention weight matrix determination unit includes:

[0133] a channel feature determination unit, configured to input the first channel feature and the second channel feature into a multilayer perceptron in the channel attention module, respectively, to obtain a third channel feature corresponding to the first channel feature and a fourth channel feature corresponding to the second channel feature;

[0134] A weight matrix determination unit is used to process the third channel features according to the activation function in the channel attention module to obtain the attention weight matrix under the maximum pooling layer, and to process the fourth channel features according to the activation function to obtain the attention weight matrix under the average pooling layer.

[0135] Based on the above device, optionally, the corrosion classification result determination module 540 includes:

[0136] A first feature to be upsampled determining unit, used for determining a first feature to be upsampled based on the attention weight matrices under different pooling layers and the features to be processed by the channels;

[0137] A corrosion classification result determination unit is used to determine a corrosion classification result corresponding to the target hardware based on the first feature to be upsampled and the feature to be sampled and processed.

[0138] On the basis of the above device, optionally, the hardware corrosion classification model further includes a first upsampling unit, a sixth convolution unit, a second upsampling unit, a seventh convolution unit, a third upsampling unit, an eighth convolution unit, a fourth upsampling unit and a ninth convolution unit, and the corrosion classification result determination unit includes:

[0139] A first feature to be spliced ​​determining unit, configured to process the first feature to be upsampled based on the first upsampling unit in the hardware corrosion classification model to obtain a first feature to be spliced;

[0140] a fifth feature to be processed determining unit, configured to obtain a fifth feature to be processed based on a fourth feature to be downsampled and the first feature to be spliced ​​in the feature to be sampled and processed;

[0141] The second feature to be upsampled is processed based on the sixth convolution unit in the hardware corrosion classification model on the fifth feature to be processed to obtain the second feature to be upsampled;

[0142] A second feature to be spliced ​​determining unit, used for processing the second feature to be upsampled based on the second upsampling unit in the hardware corrosion classification model to obtain a second feature to be spliced;

[0143] a sixth feature to be processed determining unit, configured to obtain a sixth feature to be processed based on the third feature to be downsampled and the second feature to be spliced ​​in the feature to be sampled and processed;

[0144] A third feature to be upsampled determining unit, used for processing the sixth feature to be processed based on the seventh convolution unit in the hardware corrosion classification model to obtain a third feature to be upsampled;

[0145] A third feature to be spliced ​​determining unit, used for processing the third feature to be upsampled based on the third upsampling unit in the hardware corrosion classification model to obtain a third feature to be spliced;

[0146] a seventh feature to be processed determining unit, configured to obtain a seventh feature to be processed based on the second feature to be downsampled and the third feature to be spliced ​​in the feature to be sampled and processed;

[0147] A fourth feature to be upsampled determining unit, used for processing the seventh feature to be processed based on the eighth convolution unit in the hardware corrosion classification model to obtain a fourth feature to be upsampled;

[0148] A fourth feature to be spliced ​​determining unit, used for processing the fourth feature to be upsampled based on the fourth upsampling unit in the hardware corrosion classification model to obtain a fourth feature to be spliced;

[0149] an eighth feature-to-be-processed determining unit, configured to obtain an eighth feature-to-be-processed based on the first feature-to-be-downsampled and the fourth feature-to-be-joined in the feature-to-be-sampled and processed;

[0150] The corrosion classification result determination subunit is used to process the eighth feature to be processed based on the ninth convolution unit in the hardware corrosion classification model to determine the corrosion classification result corresponding to the target hardware.

[0151] On the basis of the above device, optionally, the device further includes:

[0152] A training sample determination unit is used to obtain a plurality of training samples, wherein the training samples include a metal fitting corrosion image and a graded corrosion annotation map corresponding to the metal fitting corrosion image; the graded corrosion annotation map includes at least one annotated corrosion grade;

[0153] A predicted corrosion classification map determining unit is used to input the hardware corrosion image in the training sample into the hardware corrosion classification model to be trained for each training sample, so as to obtain a predicted corrosion classification map;

[0154] A prediction result determination unit, configured to determine, from the predicted corrosion classification map, a prediction result corresponding to the annotated corrosion grade in the graded corrosion annotation map in the training sample;

[0155] A loss value determination unit, used to determine a loss value based on the prediction result and a loss function in the hardware corrosion classification model;

[0156] The model training unit is used to modify the model parameters in the hardware corrosion classification model based on the loss value to obtain a pre-trained hardware corrosion classification model.

[0157] On the basis of the above device, optionally, the device further includes:

[0158] A test sample determination unit is used to obtain a plurality of test samples, wherein the test samples include a metal fitting corrosion image and a corrosion grade annotation map corresponding to the metal fitting corrosion image; the corrosion grade annotation map includes a plurality of first pixel points and annotated corrosion grades corresponding to the first pixel points;

[0159] A test corrosion classification map determining unit is used to input the hardware corrosion image in the test sample into a pre-trained hardware corrosion classification model for each test sample to obtain a test corrosion classification map; the test corrosion classification map includes a plurality of second pixel points and corrosion prediction results corresponding to the second pixel points;

[0160] An index value determination unit is used to determine an index value corresponding to at least one preset evaluation index based on a plurality of first pixel points in the corrosion grading annotation map in the test sample and the annotated corrosion grades corresponding to the first pixel points, and a plurality of second pixel points in the corresponding test corrosion classification map and the corrosion prediction results corresponding to the second pixel points; wherein the preset evaluation index includes at least an overall accuracy index, an average class intersection-over-union ratio index and an average pixel accuracy index;

[0161] The model test determination unit is used to obtain a trained hardware corrosion classification model when the indicator value meets the preset conditions.

[0162] The device for classifying metal corrosion provided in the embodiment of the present invention can execute the method for classifying metal corrosion provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0163] Embodiment 6

[0164] Fig. 9 It is a schematic diagram of the structure of an electronic device that implements the classification method of hardware corrosion of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0165] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0166] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0167] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a classification method for metal corrosion.

[0168] In some embodiments, the classification method for metal corrosion can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the classification method for metal corrosion described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the classification method for metal corrosion in any other appropriate manner (for example, by means of firmware).

[0169] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0171] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0173] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0175] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the classification method for metal fitting corrosion provided in any embodiment of the present invention.

[0176] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0177] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0178] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A classification method for metal fittings corrosion, characterized in that: include: Acquire the image to be classified corresponding to the target hardware; The image to be classified is input into the trained hardware corrosion classification model to obtain the hardware features to be used; wherein the hardware features to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified by the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model; Processing the channel-processing features to be processed to obtain attention weight matrices under different pooling layers; Based on the attention weight matrices under different pooling layers, the features to be processed in the channels, and the features to be processed by sampling, the corrosion classification results corresponding to the target hardware are determined.

2. The method according to claim 1, characterized in that The first convolution module includes a first convolution unit, a first downsampling unit, a second convolution unit, a second downsampling unit, a third convolution unit, a third downsampling unit, and a fourth convolution unit; the second convolution module includes a fourth downsampling unit and a fifth convolution unit; the image to be classified is input into the trained hardware corrosion classification model to obtain the features of the hardware to be used, including: Processing the image to be classified based on the first convolution unit in the first convolution module to obtain a first feature to be downsampled; Processing the first to-be-downsampled feature based on the first downsampling unit in the first convolution module to obtain a first to-be-processed feature; Processing the first feature to be processed based on the second convolution unit in the first convolution module to obtain a second feature to be downsampled; Processing the second to-be-downsampled feature based on the second downsampling unit in the first convolution module to obtain a second to-be-processed feature; Processing the second feature to be processed based on the third convolution unit in the first convolution module to obtain a third feature to be downsampled; Processing the third to-be-downsampled feature based on the third downsampling unit in the first convolution module to obtain a third to-be-processed feature; Processing the third feature to be processed based on the fourth convolution unit in the first convolution module to obtain a fourth feature to be downsampled; Processing the fourth to-be-downsampled feature based on the fourth downsampling unit in the second convolution module to obtain a fourth to-be-processed feature; The fourth feature to be processed is processed based on the fifth convolution unit in the second convolution module to obtain the feature to be channel processed, and the feature to be sampled and processed is determined based on at least one of the first feature to be downsampled, the second feature to be downsampled, the third feature to be downsampled, and the fourth feature to be downsampled.

3. The method according to claim 1, characterized in that: The hardware corrosion classification model also includes a channel attention module, which includes a maximum pooling layer and an average pooling layer; the processing of the channel-processed features to be processed to obtain attention weight matrices under different pooling layers includes: Processing the channel-processed feature based on the maximum pooling layer in the channel attention module to obtain a first channel feature; Processing the channel-processed feature based on the average pooling layer in the channel attention module to obtain a second channel feature; Based on the first channel features, an attention weight matrix under the maximum pooling layer is determined, and based on the second channel features, an attention weight matrix under the average pooling layer is determined.

4. The method according to claim 3, characterized in that The channel attention module further includes a multilayer perceptron and an activation function; the determining of the attention weight matrix under the maximum pooling layer based on the first channel feature, and the determining of the attention weight matrix under the average pooling layer based on the second channel feature, includes: Inputting the first channel feature and the second channel feature into the multilayer perceptron in the channel attention module respectively to obtain a third channel feature corresponding to the first channel feature and a fourth channel feature corresponding to the second channel feature; The third channel feature is processed according to the activation function in the channel attention module to obtain the attention weight matrix under the maximum pooling layer, and the fourth channel feature is processed according to the activation function to obtain the attention weight matrix under the average pooling layer.

5. The method according to claim 1 or 3, characterized in that: The step of determining the corrosion classification result corresponding to the target hardware based on the attention weight matrix under different pooling layers, the channel processing features and the sampling processing features includes: Determine a first feature to be upsampled based on the attention weight matrices under different pooling layers and the features to be processed by the channels; Based on the first feature to be sampled and the feature to be sampled and processed, a corrosion classification result corresponding to the target hardware is determined.

6. The method according to claim 5, characterized in that The hardware corrosion classification model further includes a first upsampling unit, a sixth upsampling unit, a second upsampling unit, a seventh upsampling unit, a third upsampling unit, an eighth upsampling unit, a fourth upsampling unit and a ninth upsampling unit. The method of determining the corrosion classification result corresponding to the target hardware based on the first feature to be upsampled and the feature to be sampled and processed includes: Processing the first feature to be upsampled based on the first upsampling unit in the hardware corrosion classification model to obtain a first feature to be spliced; Based on a fourth feature to be downsampled in the features to be sampled and processed and the first feature to be concatenated, a fifth feature to be processed is obtained; Processing the fifth feature to be processed based on the sixth convolution unit in the hardware corrosion classification model to obtain a second feature to be upsampled; Processing the second to-be-upsampled feature based on the second upsampling unit in the hardware corrosion classification model to obtain a second to-be-joined feature; Based on the third feature to be downsampled and the second feature to be spliced ​​in the feature to be sampled and processed, a sixth feature to be processed is obtained; Processing the sixth feature to be processed based on the seventh convolution unit in the hardware corrosion classification model to obtain a third feature to be upsampled; Processing the third feature to be upsampled based on the third upsampling unit in the hardware corrosion classification model to obtain a third feature to be spliced; Based on the second feature to be downsampled and the third feature to be spliced ​​in the feature to be sampled and processed, obtaining a seventh feature to be processed; Processing the seventh feature to be processed based on the eighth convolution unit in the hardware corrosion classification model to obtain a fourth feature to be upsampled; Processing the fourth feature to be upsampled based on the fourth upsampling unit in the hardware corrosion classification model to obtain a fourth feature to be spliced; Based on the first to-be-downsampled feature and the fourth to-be-joined feature in the to-be-sampled processing features, an eighth to-be-processed feature is obtained; The eighth feature to be processed is processed based on the ninth convolution unit in the hardware corrosion classification model to determine a corrosion classification result corresponding to the target hardware.

7. The method according to claim 1, characterized in that Before inputting the image to be classified into the trained hardware corrosion classification model to obtain the features of the hardware to be used, the method further includes: Acquire a plurality of training samples, wherein the training samples include a hardware corrosion image and a graded corrosion annotation map corresponding to the hardware corrosion image; the graded corrosion annotation map includes at least one annotated corrosion grade; For each of the training samples, the hardware corrosion image in the training sample is input into the hardware corrosion classification model to be trained to obtain a predicted corrosion classification map; Determine, from the predicted corrosion classification map, a prediction result corresponding to the annotated corrosion grade in the graded corrosion annotation map in the training sample; Determining a loss value based on the prediction result and the loss function in the hardware corrosion classification model; Model parameters in the hardware corrosion classification model are modified based on the loss value to obtain a pre-trained hardware corrosion classification model.

8. The method according to claim 7, characterized in that After obtaining the pre-trained hardware corrosion classification model, the method further includes: Acquire multiple test samples, wherein the test samples include a hardware corrosion image and a corrosion grade annotation map corresponding to the hardware corrosion image; the corrosion grade annotation map includes multiple first pixel points and annotated corrosion grades corresponding to the first pixel points; For each of the test samples, the hardware corrosion image in the test sample is input into the pre-trained hardware corrosion classification model to obtain a test corrosion classification map; the test corrosion classification map includes a plurality of second pixel points and corrosion prediction results corresponding to the second pixel points; Based on a plurality of first pixel points in the corrosion grading annotation map in the test sample and the annotated corrosion grades corresponding to the first pixel points, and a plurality of second pixel points in the corresponding test corrosion classification map and the corrosion prediction results corresponding to the second pixel points, an indicator value corresponding to at least one preset evaluation indicator is determined; wherein the preset evaluation indicator includes at least an overall accuracy indicator, an average class intersection-over-union ratio indicator, and an average pixel accuracy indicator; When the index value meets the preset conditions, a trained hardware corrosion classification model is obtained.

9. A classification device for metal fittings corrosion, characterized in that: include: The module for acquiring images to be classified is used to acquire images to be classified corresponding to target hardware; The hardware feature determination module to be used is used to input the image to be classified into the trained hardware corrosion classification model to obtain the hardware features to be used; wherein the hardware features to be used include features to be sampled and processed and features to be processed through channels; the features to be sampled are obtained by processing the image to be classified by the first convolution module in the hardware corrosion classification model; the features to be processed through channels are obtained by processing the features to be sampled by the second convolution module in the hardware corrosion classification model; An attention weight matrix determination module is used to process the features to be processed by the channel to obtain attention weight matrices under different pooling layers; The corrosion classification result determination module is used to determine the corrosion classification result corresponding to the target hardware based on the attention weight matrix under the different pooling layers, the channel processing features and the sampling processing features.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for classifying corrosion of metal fittings according to any one of claims 1 to 8.

Images