Rust detection model construction method and corrosion detection method

Abstract

The present invention provides a method for building a corrosion detection model and a corrosion detection method. The method for building a corrosion detection model includes: obtaining model training data, the model training data includes multiple metal equipment images, each metal equipment image is marked with a corrosion area; based on the model training data, 2 Net network model is trained to obtain the corrosion detection model, U 2 The top layer of the Net network model is composed of an encoder and a decoder forming a U-shaped network structure, and the encoder and the decoder are respectively composed of multiple layers of Swin Transformer units forming a U-shaped network structure. Through the present invention, the Swin Transformer unit is used to perform local and global feature learning, extract multi-scale features of each stage and aggregate multi-level features of each stage, so as to improve the accuracy of the rust detection model in detecting rusted areas.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image detection, and in particular to a corrosion detection model construction method and a corrosion detection method. Background Art

[0002] As the gathering center of electricity generated by offshore wind power, daily inspection of offshore substations is an inevitable part of offshore wind power operation and maintenance. Offshore substations and their power equipment are mostly metal steel frame structures, and the entire substation is in a humid salt spray environment at sea, which accelerates the corrosion of metal equipment. Corrosion detection of the appearance of the substation and its equipment has become a key step in the inspection of offshore substations.

[0003] Routine manual inspections are dangerous and arduous, and intelligent inspections based on machine vision have become an inevitable trend in offshore wind power inspections. The degree of corrosion of different metal equipment varies, which is reflected in the size and color of the rusted area. This increases the difficulty of traditional image processing corrosion detection methods, and the accuracy of the corrosion detection model needs to be improved. Summary of the invention

[0004] In order to improve the accuracy of the corrosion detection model, the present invention provides a corrosion detection model construction method and a corrosion detection method.

[0005] In a first aspect, the present invention provides a method for constructing a corrosion detection model, the method comprising:

[0006] Obtain model training data, where the model training data includes a plurality of metal equipment images, each of which is marked with a rusted area;

[0007] Based on the model training data, U 2 Net network model is trained to obtain the corrosion detection model, U 2 The top layer of the Net network model consists of an encoder and a decoder forming a U-shaped network structure, and the encoder and decoder each consist of a U-shaped network structure consisting of multiple layers of SwinTransformer units.

[0008] Through the above method, Swin Transformer and U 2 Net network model is combined to form a two-layer nested U-shaped structure. The Swin Transformer unit is used to perform local and global feature learning, extract multi-scale features at each stage and aggregate multi-level features at each stage. This can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of the corrosion detection model in detecting rusted areas.

[0009] In combination with the first aspect, in a first embodiment of the first aspect, U 2The top layer of the Net network model includes at least two layers of encoders, at least one layer of decoders, and a saliency map fusion model. The encoder and the corresponding decoder are skipped. 2 The last encoder layer and each decoder in the Net network model are connected to the saliency map fusion model respectively.

[0010] In combination with the first aspect or the first embodiment of the first aspect, in a second embodiment of the first aspect, U 2 The encoder and decoder in the top layer of the Net network model include: encoding module, decoding module, bottleneck layer, encoding module, bottleneck layer and decoding module are connected in sequence.

[0011] The encoding module, decoding module and bottleneck layer all include at least one layer of Swin Transformer unit;

[0012] The Swin Transformer unit in the encoding module is jump-connected to the corresponding Swin Transformer unit in the decoding module.

[0013] In combination with the first aspect, in a third embodiment of the first aspect, obtaining model training data includes:

[0014] Get initial training data;

[0015] Convert the metal device image in the initial training data from the RGB color space to the HSV color space to obtain HSV color space data;

[0016] Selecting a rust color category and a corresponding color threshold based on HSV color space data to perform color segmentation and generate a mask image of at least one rust color;

[0017] The mask images obtained by segmentation corresponding to the rust color category are merged to obtain the rust area labeled based on HSV color segmentation;

[0018] The rusted area labeled based on HSV color segmentation is calibrated to obtain model training data.

[0019] Through the above embodiment, the initial training data is annotated using HSV color segmentation to obtain a plurality of rust regions with different rust colors, thereby enhancing the accuracy of training data annotation, improving annotation efficiency, and reducing the workload of manual annotation.

[0020] In combination with the first aspect, in a fourth embodiment of the first aspect, based on the model training data, U 2 Net network model is trained to obtain a corrosion detection model, including: 2Net network model is trained until the loss value of the preset loss function meets the preset loss condition, and the corrosion detection model is obtained.

[0021] In a second aspect, the present invention provides a corrosion detection method, the method comprising:

[0022] Acquire an image of a metal device to be inspected;

[0023] Performing corrosion detection on the metal equipment image to be detected based on the corrosion detection model to obtain a binary image, wherein the corrosion detection model is established by the corrosion detection model construction method in the first aspect or any embodiment of the first aspect;

[0024] Determine the rusted area in the metal equipment image based on the binary image.

[0025] Through the above method, the Swin Transformer unit in the corrosion detection model is used to perform local and global feature learning, extract multi-scale features of each stage and aggregate multi-level features of each stage, which can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of detecting rusted areas.

[0026] In a third aspect, the present invention provides a corrosion detection model construction device, the device comprising:

[0027] A first acquisition module is used to acquire model training data, wherein the model training data includes a plurality of metal equipment images, each of which is marked with a rusted area;

[0028] Establish a module for U based on model training data 2 Net network model is trained to obtain the corrosion detection model, U 2 The top layer of the Net network model consists of an encoder and a decoder forming a U-shaped network structure, and the encoder and decoder each consist of a U-shaped network structure consisting of multiple layers of Swin Transformer units.

[0029] Through the above device, Swin Transformer and U 2 Net network model is combined to form a two-layer nested U-shaped structure. The Swin Transformer unit is used to perform local and global feature learning, extract multi-scale features at each stage and aggregate multi-level features at each stage. This can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of the corrosion detection model in detecting rusted areas.

[0030] In a fourth aspect, the present invention provides a corrosion detection device, the device comprising:

[0031] A second acquisition module is used to acquire an image of a metal device to be detected;

[0032] A detection module, used to perform corrosion detection on the metal equipment image to be detected based on a corrosion detection model to obtain a binary image, wherein the corrosion detection model is established by the corrosion detection model construction method in the first aspect or any embodiment of the first aspect;

[0033] The determination module is used to determine the rust area in the metal equipment image based on the binary image.

[0034] Through the above device, the Swin Transformer unit in the corrosion detection model is used to perform local and global feature learning, extract multi-scale features of each stage and aggregate multi-level features of each stage, which can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of detecting rusted areas.

[0035] In a fifth aspect, the present invention further provides a computer device, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the rust detection model building method of the first aspect or any embodiment of the first aspect, or the steps of the rust detection method of the second aspect by executing the computer instructions.

[0036] In a sixth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for constructing a rust detection model in the first aspect or any embodiment of the first aspect, or the steps of the rust detection method in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of a method for building a corrosion detection model according to an exemplary embodiment;

[0039] Figure 2 In one example, U 2 Net network model top-level structure diagram;

[0040] Figure 3 In one example, U 2 Schematic diagram of the structure of RSTU-2 in Net network model;

[0041] Figure 4is a flow chart of a corrosion detection method proposed according to an exemplary embodiment;

[0042] FIG5( a ) is an image of metal equipment corrosion in one example;

[0043] FIG5( b ) is a binary image of a rusted area corresponding to a rusted image of a metal device in an example;

[0044] FIG6( a ) is an image of metal equipment corrosion in another example;

[0045] FIG6( b ) is a binary image of a rusted area corresponding to a rusted image of a metal device in another example;

[0046] Figure 7 is a structural block diagram of a corrosion detection model building device proposed according to an exemplary embodiment;

[0047] Figure 8 is a structural block diagram of a corrosion detection device proposed according to an exemplary embodiment;

[0048] Fig. 9 It is a schematic diagram of the hardware structure of a computer device proposed according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0051] In order to improve the accuracy of the corrosion detection model, the present invention provides a corrosion detection model construction method and a corrosion detection method.

[0052] Figure 1 FIG. 1 is a flowchart of a method for constructing a corrosion detection model according to an exemplary embodiment. Figure 1 As shown, the corrosion detection model construction method includes the following steps:

[0053] Step S101: acquiring model training data, wherein the model training data includes a plurality of metal equipment images, each of which is marked with a rusted area.

[0054] Specifically, the initial data before the model training data is labeled can be the appearance images of metal equipment collected by a camera or other equipment, and the collected appearance images include rust images and non-rust images. Among them, the rust images include rust areas with different rust degrees and different natural conditions. Different rust degrees are manifested in differences in size, color, etc. Different natural conditions refer to differences in light, temperature, etc., so that the training data contains as many metal equipment images under various detection environments as possible.

[0055] In an optional embodiment, the initial data is expanded using at least one of random image shearing, random image rotation, image mirroring, and image brightness transformation to obtain expanded training data. By expanding the initial data, the diversity of the training data is increased, and the robustness of the model training is enhanced.

[0056] Step S102: Based on the model training data, 2 Net network model is trained to obtain the corrosion detection model, U 2 The top layer of the Net network model consists of an encoder and a decoder forming a U-shaped network structure, and the encoder and decoder each consist of a U-shaped network structure consisting of multiple layers of Swin Transformer units.

[0057] Specifically, U 2 The number of encoders and decoders in the top-level structure of the Net network model can be adaptively adjusted as needed. Similarly, the number of layers of Swin Transformer units in each encoder or decoder can also be configured according to the spatial resolution of the input feature map.

[0058] Through the above method, Swin Transformer and U 2 Net network model is combined to form a two-layer nested U-shaped structure. The Swin Transformer unit is used to perform local and global feature learning, extract multi-scale features at each stage and aggregate multi-level features at each stage. This can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of the corrosion detection model in detecting rusted areas.

[0059] In one example, the model training data in step S101 above can be obtained by HSV color segmentation, and the specific steps are as follows:

[0060] First, initial training data is obtained, and the metal device images in the initial training data are converted from the RGB color space to the HSV color space to obtain HSV color space data.

[0061] The original RGB image corresponds to the raw data output by the camera sensor, while the HSV image is more consistent with human intuitive vision. The annotation results obtained based on HSV image segmentation are more consistent with the manual annotation results. Therefore, the RGB image can be converted to an HSV image first, and then color segmentation can be performed. The HSV image contains three-channel data of the same size: H channel, S channel, and V channel. The pixel value of the H channel represents the hue, ranging from 0 to 360; the pixel value of the S channel represents the saturation of the image, ranging from 0 to 1V; the channel pixel value represents the brightness of the image, and the value range is also 0 to 1. The larger the value, the brighter it is. Exemplarily, an RGB image can be converted to an HSV image using the following formula:

[0062] V = max(R, G, B)

[0063]

[0064]

[0065] In the formula, R, G, and B are the three-channel component values ​​corresponding to the RGB color space, and H, S, and V are the three-channel component values ​​corresponding to the HSV color space.

[0066] Secondly, based on the HSV color space data, the rust color category and the corresponding color threshold are selected for color segmentation to generate a mask image of at least one rust color.

[0067] In an optional embodiment, the metal corrosion image is shown in the three most common color categories of brown, brown and yellow in color appearance. Therefore, according to the value range of brown, brown and yellow in the HSV color space, color segmentation can be performed in the HSV color space to generate mask images of the three corrosion colors.

[0068] Again, the mask images obtained by segmentation corresponding to the rust color category are merged to obtain the rust area labeled based on HSV color segmentation.

[0069] Finally, the rusted area annotated based on HSV color segmentation is calibrated to obtain the model training data.

[0070] Due to the complexity of the color of the rusted area, in order to avoid errors when using HSV color segmentation annotation, the rusted area annotated by HSV color segmentation needs to be further calibrated to ensure the accuracy of the model training data. For example, the rusted area can be roughly annotated by HSV color segmentation, and then manually finely annotated to finally obtain the data for model training. Combining HSV color segmentation with manual standards can not only reduce the workload of manual annotation, but also help to enhance the accuracy of annotation and improve the efficiency of annotation.

[0071] In an optional embodiment, the labelme tool is used for manual fine calibration: first, the HSV color segmentation and annotation file is converted into a JSON file, and then the Labelme tool is used to open the original rust image and the JSON file corresponding to the image, and the label is manually calibrated through the Labelme tool. The calibration content includes deleting the mislabeled rust area and adding the unlabeled rust area. Finally, the calibrated JSON file is converted into a PNG label image to obtain the final model training data.

[0072] In one example, U 2 The top layer of the Net network model includes at least two layers of encoders, at least one layer of decoders, and a saliency map fusion model. The encoder and the corresponding decoder are skipped. 2 The last encoder layer and each decoder in the Net network model are connected to the saliency map fusion model respectively.

[0073] Figure 2 U 2 The top layer in the Net network model consists of a 4-layer encoder, a 3-layer decoder and a saliency map fusion model, where the 4-layer encoder and the 3-layer decoder form a U-shaped network structure.

[0074] In each layer of the encoder, there is a U-shaped network structure composed of Swin Transformer units, that is, an RSTU-L network structure. L represents the number of layers of Swin Transformer units in the encoder or decoder in the RSTU network structure. L can be configured according to the spatial resolution of the input feature map. In each layer of the encoder, the feature map is downsampled, that is, the scale of the output feature map is reduced after each layer of the encoder. In encoder 1, RSTU-4 composed of 4 layers of Swin Transformer units is used; in encoder 2, RSTU-3 composed of 3 layers of Swin Transformer units is used; in encoders 3 and 4, the resolution of features is relatively low, and further downsampling of these feature maps will result in the loss of useful context. For example, encoders 3 and 4 both use RSTU-2 composed of 2 layers of Swin Transformer units.

[0075] In each layer of decoder, the decoding stage has a similar structure to the encoding stage. In decoder 3, RSTU-2 is used, and RSTU-2 and RSTU-3 are used in decoder 2 and decoder 1, respectively. Each layer of decoder takes the concatenation of the upsampled feature map from the previous level and the feature map from its corresponding encoder as input to achieve the fusion of shallow and deep information, providing more semantic information for the decoding process.

[0076] The saliency map fusion model is used to generate saliency probability maps. Four saliency probability maps are generated from encoder 4, decoder 3, decoder 2 and decoder 1 through 3x3 convolution and Sigmoid function. The logical maps of the four saliency probability maps are upsampled to the same size as the input image and fused through cascade operation. Then, a 1x1 convolution layer and a Sigmoid function are used to generate the final saliency probability map to obtain the corrosion detection result.

[0077] In an alternative embodiment, U 2 The encoder and decoder in the top layer of the Net network model include: encoding module, decoding module, bottleneck layer, encoding module, bottleneck layer and decoding module are connected in sequence, and the encoding module, decoding module and bottleneck layer each include at least one layer of Swin Transformer unit; the Swin Transformer unit in the encoding module is jump-connected with the Swin Transformer unit in the corresponding decoding module. Through the jump connection, the multi-scale features from the encoding module are fused with the up-sampled features, and the shallow features are connected with the deep features to reduce the spatial information loss caused by downsampling.

[0078] Figure 3 The schematic diagram of the RSTU-2 is composed of two layers of Swin Transformer units. Figure 3 As shown in Figure 1, RSTU-2 consists of an encoding module, a decoding module, and a bottleneck layer. Both the encoding module and the decoding module are composed of two layers of Swin Transformer units, and the bottleneck layer is composed of one layer of Swin Transformer units.

[0079] In the encoding module, the Swin Transformer unit is responsible for feature representation learning. In addition to the Swin Transformer unit, it also includes a patch merging layer, which is responsible for downsampling and increasing the number of channels.

[0080] The structure of the decoding module is similar to that of the encoding module. In the decoding module, in addition to the Swin Transformer unit, it also includes a patch expanding layer, and the negative value of the patch expansion layer upsamples the extracted deep features.

[0081] In the bottleneck module, since the Transformer is too deep and convergence is difficult, a 1-layer SwinTransformer unit is used to construct the bottleneck layer to learn deep feature representation. At the bottleneck layer, the feature dimension and resolution of the image remain unchanged.

[0082] In one example, in the process of building a corrosion detection model, it is also necessary to design a loss function, that is, to calculate the loss function of U based on the model training data. 2 Net network model is trained until the loss value of the preset loss function meets the preset loss condition, and the corrosion detection model is obtained.

[0083] In an optional embodiment, a multi-supervision algorithm is used to construct the loss function. 2 The output of the Net network model not only includes the final feature map, but also the feature maps of different scales output by the bottom-level encoder and all decoders. Therefore, when constructing the loss function, it is necessary to supervise not only the network output but also the intermediate fusion feature maps. The loss function is expressed as follows:

[0084]

[0085] Among them, M represents the number of feature maps output by the lowest encoder and all decoders, l side represents the loss value of each feature map output by the lowest encoder and all decoders, l fuse Represents the loss value of the final feature map, w side Represents the weight parameter corresponding to each feature map output by the lowest encoder and all decoders, w fuse Represents the weight parameter of the final feature map. Figure 2 U 2 Net network model as an example, M is 4. Exemplarily, the loss value is calculated using standard binary cross entropy.

[0086] The preset loss condition is that when the loss value of the current model calculated according to the loss function is less than the loss value set according to the need, the current model is determined to be the final corrosion detection model.

[0087] Figure 4 FIG. 1 is a flow chart of a corrosion detection method according to an exemplary embodiment. Figure 4 As shown, the method comprises the following steps:

[0088] Step S401: Acquire an image of a metal device to be detected.

[0089] Step S402: performing corrosion detection on the metal equipment image to be detected based on the corrosion detection model to obtain a binary image. The corrosion detection model is established by the corrosion detection model construction method provided in the above embodiment.

[0090] Specifically, the grayscale value of the pixel in the binary image is set to 0 or 255, that is, the image presents an obvious visual effect of only black and white.

[0091] Step S403: determining the rusted area in the metal equipment image based on the binary image.

[0092] In one example, the rust degree of the metal equipment may be evaluated based on the area or shape of the rusted area.

[0093] Through the above method, the Swin Transformer unit in the corrosion detection model is used to perform local and global feature learning, extract multi-scale features of each stage and aggregate multi-level features of each stage, which can effectively detect metal equipment with different degrees of corrosion and improve the accuracy of detecting rusted areas.

[0094] Figure 5(a) and Figure 5(b) are the rust image of metal equipment and the binary image of the corresponding rust area, and Figure 6(a) and Figure 6(b) are the rust image of another metal equipment and the binary image of the corresponding rust area. The experimental data show that this method can accurately detect the rust area with a detection accuracy of 100%.

[0095] Based on the same inventive concept, the embodiment of the present invention also provides a corrosion detection model construction device, such as Figure 7 As shown, the device comprises:

[0096] The first acquisition module 701 is used to acquire model training data, which includes multiple metal equipment images, each of which is marked with a rusted area. For details, please refer to the description of step S101 in the above embodiment, which will not be repeated here.

[0097] Establishing module 702, for based on model training data, U 2 Net network model is trained to obtain the corrosion detection model, U 2 The top layer of the Net network model is composed of an encoder and a decoder to form a U-shaped network structure, and the encoder and the decoder are respectively composed of multiple layers of Swin Transformer units to form a U-shaped network structure. For details, please refer to the description of step S102 in the above embodiment, which will not be repeated here.

[0098] In one example, in the establishment module 702, U 2 The top layer of the Net network model includes at least two layers of encoders, at least one layer of decoders, and a saliency map fusion model. The encoder and the corresponding decoder are skipped. 2 The last layer encoder and each decoder in the Net network model are respectively connected to the saliency map fusion model. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0099] In yet another example, in the establishment module 702, U 2The encoder and decoder in the top layer of the Net network model include: encoding module, decoding module, bottleneck layer, encoding module, bottleneck layer and decoding module are connected in sequence.

[0100] The encoding module, decoding module and bottleneck layer each include at least one Swin Transformer unit;

[0101] The Swin Transformer unit in the encoding module is jump-connected with the Swin Transformer unit in the corresponding decoding module. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0102] In one example, the first acquisition module 701 includes:

[0103] The acquisition submodule is used to acquire initial training data. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0104] The conversion submodule is used to convert the initial training data from the RGB color space to the HSV color space to obtain HSV training data. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0105] The generation submodule is used to perform color segmentation based on the HSV training data to generate a mask image of at least one rust color. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0106] The merging submodule is used to merge the mask images to obtain the rusted area marked based on HSV color segmentation. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0107] The calibration submodule is used to calibrate the rusted area marked based on HSV color segmentation to obtain model training data. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0108] In another example, the apparatus further includes:

[0109] The supervision module is used to supervise U based on the model training data. 2 Net network model is trained until the loss value of the preset loss function meets the preset loss condition to obtain the corrosion detection model. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0110] The embodiment of the present invention also provides a corrosion detection device, such as Figure 8 As shown, the device comprises:

[0111] The second acquisition module 801 is used to acquire the image of the metal device to be detected. For details, please refer to the description of step S401 in the above embodiment, which will not be repeated here.

[0112] The detection module 802 is used to perform corrosion detection on the metal equipment image to be detected based on the corrosion detection model to obtain a binary image. The corrosion detection model is established by the corrosion detection model construction method provided in the above embodiment. For details, please refer to the description of step S402 in the above embodiment, which will not be repeated here.

[0113] The determination module 803 is used to determine the rusted area in the metal equipment image based on the binary image. For details, please refer to the description of step S403 in the above embodiment, which will not be repeated here.

[0114] The specific limitations and beneficial effects of the above-mentioned device can be found in the above-mentioned limitations on the corrosion detection model construction method and the corrosion detection method, which will not be repeated here. The above-mentioned modules can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0115] Fig. 9 FIG. 1 is a schematic diagram of a hardware structure of a computer device according to an exemplary embodiment. Fig. 9 As shown, the device includes one or more processors 910 and a memory 920, and the memory 920 includes a persistent memory, a volatile memory, and a hard disk. Fig. 9 A processor 910 is taken as an example. The device may also include: an input device 930 and an output device 940.

[0116] The processor 910, the memory 920, the input device 930 and the output device 940 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0117] The processor 910 may be a central processing unit (CPU). The processor 910 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or the processor may be any conventional processor.

[0118] The memory 920 is a non-transient computer-readable storage medium, including persistent memory, volatile memory and hard disk, which can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the corrosion detection model construction method and the program instructions / modules corresponding to the corrosion detection method in the embodiment of the present application. The processor 910 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 920, that is, realizing any of the above-mentioned corrosion detection model construction methods and corrosion detection methods.

[0119] The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data required for use, etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 920 may optionally include a memory remotely arranged relative to the processor 910, and these remote memories may be connected to the data processing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The input device 930 can receive input digital or character information and generate signal input related to user settings and function control. The output device 940 can include display devices such as display screens.

[0121] One or more modules are stored in the memory 920, and when executed by one or more processors 910, the execution is as follows: Figure 1 The method shown.

[0122] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to Figure 1 Related description of the illustrated embodiment.

[0123] The embodiment of the present invention also provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the construction and detection methods in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0124] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0125] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for constructing a corrosion detection model, characterized in that: The method comprises: Acquire model training data, wherein the model training data includes a plurality of metal equipment images, each of which is marked with a rusted area; Based on the model training data, U 2 Net network model is trained to obtain a corrosion detection model. 2 The top layer of the Net network model is composed of an encoder and a decoder forming a U-shaped network structure, and the encoder and the decoder are respectively composed of multiple layers of Swin Transformer units forming a U-shaped network structure; The U 2 The top layer of the U2Net network model includes 4 encoders, 3 decoders and a saliency map fusion model, the encoders and corresponding decoders are jump-connected, the last encoder and each decoder in the U2Net network model are respectively connected to the saliency map fusion model, wherein the 4 encoders and 3 decoders form a U-shaped network structure; in the first encoder, RSTU-4 consisting of 4 layers of Swin Transformer units is used; in the second encoder, RSTU-3 consisting of 3 layers of Swin Transformer units is used; in the third encoder and the fourth encoder, RSTU-2 is used, in which the Swin Transformer units in the encoding module and the Swin Transformer units in the decoding module are both 2 layers; in the third decoder, RSTU-2 is used, and RSTU-3 and RSTU-2 are used in the first decoder and the second decoder respectively; The U 2 The encoder and decoder in the top layer of the Net network model include: an encoding module, a decoding module, and a bottleneck layer. The encoding module, the bottleneck layer, and the decoding module are connected in sequence. The encoding module, the decoding module and the bottleneck layer each include at least one layer of the Swin Transformer unit; The Swin Transformer unit in the encoding module is jump-connected with the Swin Transformer unit in the corresponding decoding module; The saliency map fusion model is used to generate saliency probability maps. Multiple saliency probability maps are generated from the last encoder and each decoder through 3x3 convolution and Sigmoid function. The logical maps of multiple saliency probability maps are upsampled to the same size as the input image and fused through cascade operation. Then, a 1x1 convolution layer and a Sigmoid function are used to generate the final saliency probability map, thereby obtaining the corrosion detection result. The loss function is constructed using a multi-supervision algorithm, U 2 The output of the Net network model includes the final feature map, the lowest encoder, and the feature maps of different scales output by all decoders. The loss function is expressed as follows: Among them, M represents the number of feature maps output by the lowest encoder and all decoders. l side Represents the loss value of each feature map output by the lowest encoder and all decoders, l fuse Represents the loss value of the final feature map, w side Represents the weight parameters corresponding to each feature map output by the lowest level encoder and all decoders, w fuse Represents the weight parameters of the final feature map.

2. The method according to claim 1, characterized in that Get model training data, including: Get initial training data; Convert the metal device image in the initial training data from the RGB color space to the HSV color space to obtain HSV color space data; Selecting a rust color category and a corresponding color threshold based on the HSV color space data to perform color segmentation and generate a mask image of at least one rust color; The mask images obtained by segmentation corresponding to the rust color category are merged to obtain the rust area labeled based on HSV color segmentation; The rusted area labeled based on HSV color segmentation is calibrated to obtain the model training data.

3. The method according to claim 1, characterized in that: Based on the model training data, U 2 Net network model is used for model training to obtain the corrosion detection model, including: Based on the model training data, the U 2 Net network model is trained until the loss value of the preset loss function meets the preset loss condition, thereby obtaining the corrosion detection model.

4. A corrosion detection method, characterized in that: The method comprises: Acquire an image of a metal device to be inspected; Performing corrosion detection on the metal equipment image to be detected based on the corrosion detection model to obtain a binary image, wherein the corrosion detection model is established by the corrosion detection model construction method described in any one of claims 1 to 3; A rusted area in the metal equipment image is determined based on the binarized image.

5. A corrosion detection model building device, characterized in that: The device comprises: A first acquisition module is used to acquire model training data, wherein the model training data includes a plurality of metal equipment images, each of which is marked with a rusted area; Establish a module for U based on model training data 2 Net network model is trained to obtain a corrosion detection model. 2 The top layer of the Net network model is composed of an encoder and a decoder forming a U-shaped network structure, and the encoder and the decoder are respectively composed of multiple layers of Swin Transformer units forming a U-shaped network structure; The U 2 The top layer of the UNet network model includes 4 encoder layers, 3 decoder layers, and a saliency map fusion model. The encoder and the corresponding decoder are skipped. 2 The last encoder and each decoder in the Net network model are respectively connected to the saliency map fusion model, wherein the 4-layer encoder and the 3-layer decoder form a U-shaped network structure; in the first-layer encoder, RSTU-4 composed of 4 layers of Swin Transformer units is used; in the second-layer encoder, RSTU-3 composed of 3 layers of Swin Transformer units is used; in the third-layer encoder and the fourth-layer encoder, RSTU-2 composed of 2 layers of Swin Transformer units is used, and in RSTU-2, the Swin Transformer units in the encoding module and the Swin Transformer units in the decoding module are both 2 layers; in the third-layer decoder, RSTU-2 is used, and RSTU-3 and RSTU-2 are used in the first-layer decoder and the second-layer decoder respectively; The U 2 The encoder and decoder in the top layer of the Net network model include: an encoding module, a decoding module, and a bottleneck layer. The encoding module, the bottleneck layer, and the decoding module are connected in sequence. The encoding module, the decoding module and the bottleneck layer each include at least one layer of the Swin Transformer unit; The Swin Transformer unit in the encoding module is jump-connected with the Swin Transformer unit in the corresponding decoding module; The saliency map fusion model is used to generate saliency probability maps. Multiple saliency probability maps are generated from the last encoder and each decoder through 3x3 convolution and Sigmoid function. The logical maps of multiple saliency probability maps are upsampled to the same size as the input image and fused through cascade operation. Then, a 1x1 convolution layer and a Sigmoid function are used to generate the final saliency probability map, thereby obtaining the corrosion detection result. The loss function is constructed using a multi-supervisory algorithm. The output of the U2Net network model includes the final feature map, the feature maps of different scales output by the bottom-level encoder and all decoders. The loss function is expressed as follows: Among them, M represents the number of feature maps output by the lowest encoder and all decoders. l side Represents the loss value of each feature map output by the lowest encoder and all decoders, l fuse Represents the loss value of the final feature map, w side Represents the weight parameters corresponding to each feature map output by the lowest level encoder and all decoders, w fuse Represents the weight parameters of the final feature map.

6. A corrosion detection device, characterized in that: The device comprises: A second acquisition module is used to acquire an image of a metal device to be detected; A detection module, used for performing corrosion detection on the metal equipment image to be detected based on a corrosion detection model to obtain a binary image, wherein the corrosion detection model is established by the corrosion detection model construction method according to any one of claims 1 to 3; A determination module is used to determine the rusted area in the metal equipment image based on the binary image.

7. A computer device, characterized in that: It comprises a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the corrosion detection model construction method described in any one of claims 1 to 3, or executes the steps of the corrosion detection method described in claim 4 by executing the computer instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing a corrosion detection model as described in any one of claims 1 to 3 is implemented, or the steps of the corrosion detection method as described in claim 4 are implemented.

Images