Semantic segmentation method, device, equipment and medium for corrosion damage of steel structure bridges

Abstract

The present invention discloses a semantic segmentation method, device, equipment and medium for rust damage of steel structure bridges, and relates to the technical field of bridge damage detection. The method comprises obtaining a rust damage picture and annotating it to generate an annotated image, cutting the rust damage picture and the annotated image into image blocks of a set size based on a set cutting size and a set cutting step length; using an improved three-branch network structure as a semantic segmentation algorithm, building a network model, and training the network model; obtaining an image to be predicted and cutting the image to be predicted into image blocks of a set size based on a set cutting size and a set cutting step length, and inputting the trained network model to obtain a prediction result; constructing a blank image, and filling the rust damage prediction result image of each image block to be predicted into the blank image based on a preset filling splicing rule, and completing the splicing of the prediction result. The present application can effectively reduce the dependence on hardware equipment, and at the same time improve the recognition ability of small target areas.

Description

Technical Field

[0001] The present application relates to the technical field of bridge disease detection, and specifically to a method, device, equipment and medium for semantic segmentation of corrosion diseases in steel structure bridges. Background Art

[0002] As an important transportation infrastructure, the structural safety of bridges is directly related to the safety of people's lives and property. The corrosion of steel structures will reduce the bearing capacity of components and increase the risk of structural failure. Through corrosion detection, potential safety problems of the structure can be discovered in time and measures can be taken to avoid accidents. However, traditional manual corrosion detection methods have problems such as low efficiency and easy omissions.

[0003] With the development of technologies such as artificial intelligence, some researchers have used image semantic segmentation technology to identify rust diseases in images. However, at present, semantic segmentation technology only works well on some small-size images. The main reason is that it requires a lot of computing resources in the calculation and reasoning process, which leads to the inability to run smoothly when training and reasoning on images with a resolution of 4K or above taken by drones due to limited hardware performance. In addition, since rust diseases account for a small proportion in large images, rust diseases are difficult to be accurately identified. Summary of the invention

[0004] The present application provides a method, device, equipment and medium for semantic segmentation of corrosion damage in steel structure bridges, which can effectively reduce the dependence on hardware equipment and improve the recognition ability of small target areas.

[0005] In a first aspect, an embodiment of the present application provides a method for semantic segmentation of corrosion damage on a steel structure bridge, the method comprising:

[0006] Obtain high-resolution corrosion disease images of bridges and annotate them to generate annotated images. Based on the set cutting size and the set cutting step length, cut both the corrosion disease images and the annotated images into image blocks of set sizes.

[0007] The improved three-branch network structure is used as the semantic segmentation algorithm to build a network model, and the network model is trained by cutting the image blocks.

[0008] Obtaining an image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the blocks into the trained network model to obtain a prediction result;

[0009] A blank image is constructed, and based on the preset filling and splicing rules, the rust disease prediction result images of each image block to be predicted are filled into the blank image to complete the splicing of the prediction results.

[0010] In combination with the first aspect, in one implementation, obtaining a high-resolution corrosion disease picture of a bridge and annotating it to generate an annotated image specifically includes:

[0011] Obtain high-resolution corrosion damage images of bridges;

[0012] Based on the image segmentation model or the active annotation method, the rust disease parts in the acquired rust disease picture are annotated to obtain an annotated image.

[0013] In combination with the first aspect, in one implementation, the step of cutting the rust disease image and the annotated image into image blocks of a set size based on the set cutting size and the set cutting step size specifically includes:

[0014] A pixel block of a first size×a first size is used as a set cutting size, and a pixel of a second size is used as a set cutting step length;

[0015] According to the cutting order from left to right and from top to bottom, the rust disease image is cut into a plurality of image blocks of first size × first size pixels, and the labeled image is cut into a plurality of image blocks of first size × first size pixels;

[0016] The second size corresponding value is half of the first size corresponding value.

[0017] In combination with the first aspect, in one embodiment, after the rust disease picture and the annotated image are cut into image blocks of a set size, the method further includes:

[0018] Obtain image blocks of the annotated image in sequence, and determine whether the rust disease pixels in the obtained image blocks are greater than a preset percentage of the pixels of the entire image block:

[0019] If yes, keep the current image block of the annotated image;

[0020] If not, the current image block of the labeled image is deleted, and the image block corresponding to the rust disease picture is deleted at the same time.

[0021] In combination with the first aspect, in one implementation, obtaining the image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the trained network model to obtain a prediction result specifically includes:

[0022] Acquire an image to be predicted, and cut the image to be predicted into a plurality of image blocks to be predicted of a first size×first size of pixels based on a set cutting size and a set cutting step size;

[0023] The image blocks to be predicted are input into the trained network model, the rust disease parts in each image block to be predicted are identified, and the rust disease prediction result image of the image block to be predicted is obtained.

[0024] In combination with the first aspect, in one implementation, the blank image is constructed, and the rust disease prediction result image of each image block to be predicted is filled into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results, specifically including:

[0025] Constructing a blank image with the same pixel size as the image to be predicted, and filling the blank image with pixels according to the first filling rule and the second filling rule in sequence to obtain a filled image;

[0026] Using the third largest pixel as the moving step, the rust disease prediction result image is filled into the filled image in order from left to right and from top to bottom;

[0027] Crop the pixel-filled part of the padded image to obtain a spliced ​​image of the prediction result;

[0028] The first filling rule is: dividing the number of pixels on the width of the image to be predicted by the first size to obtain a first remainder, and the difference between the first size and the first remainder is the number of pixels to be filled in the width direction of the blank image; dividing the number of pixels on the height of the image to be predicted by the first size to obtain a second remainder, and the difference between the first size and the second remainder is the number of pixels to be filled in the height direction of the blank image, and performing pixel filling on the blank image to obtain a pre-filled image;

[0029] The second filling rule is: fill the third size pixels around the pre-filled image to obtain a filled image, and the corresponding value of the third size is half of the corresponding value of the second size.

[0030] In combination with the first aspect, in one implementation, the improved three-branch network structure is used as a semantic segmentation algorithm to build a network model, and the network model is trained by cutting the image blocks, specifically including:

[0031] A pixel attention enhancement module is added to the I branch of PIDNet to obtain an improved three-branch network structure;

[0032] Use the improved three-branch network structure as the semantic segmentation algorithm to build a network model;

[0033] The network model is trained using the cut rust disease pictures and image blocks of the labeled images.

[0034] In a second aspect, an embodiment of the present application provides a semantic segmentation device for corrosion damage of a steel structure bridge, wherein the semantic segmentation device for corrosion damage of a steel structure bridge comprises:

[0035] An annotation module is used to obtain high-resolution corrosion disease images of bridges and annotate them to generate annotated images. Based on a set cutting size and a set cutting step, the corrosion disease images and the annotated images are cut into image blocks of a set size.

[0036] The training module is used to use the improved three-branch network structure as a semantic segmentation algorithm to build a network model and train the network model using image blocks obtained by cutting;

[0037] A prediction module is used to obtain an image to be predicted and cut the image to be predicted into image blocks of a set size based on a set cutting size and a set cutting step size, and input the blocks into a trained network model to obtain a prediction result;

[0038] The splicing module is used to construct a blank image and fill the rust disease prediction result image of each image block to be predicted into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results.

[0039] In the third aspect, an embodiment of the present application provides a semantic segmentation device for rust damage in steel structure bridges, wherein the semantic segmentation device for rust damage in steel structure bridges comprises a processor, a memory, and a semantic segmentation program for rust damage in steel structure bridges stored in the memory and executable by the processor, wherein when the semantic segmentation program for rust damage in steel structure bridges is executed by the processor, the steps of the above-mentioned semantic segmentation method for rust damage in steel structure bridges are implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a semantic segmentation program for rust damage to steel structure bridges is stored. When the semantic segmentation program for rust damage to steel structure bridges is executed by a processor, the steps of the above-mentioned method for semantic segmentation of rust damage to steel structure bridges are implemented.

[0041] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0042] By cutting ultra-high-resolution images into small images for training and reasoning, the dependence on hardware equipment is effectively reduced, while also improving the ability to recognize small target areas. It can be well applied to UAV-based steel structure bridge corrosion disease detection projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the semantic segmentation method for corrosion damage of steel structure bridges in this application;

[0044] Figure 2 Schematic diagram of image pixel filling and image stitching;

[0045] Figure 3This is a schematic diagram of the functional modules of the semantic segmentation device for corrosion damage of steel structure bridges in this application;

[0046] Figure 4 This is a schematic diagram of the hardware structure of the semantic segmentation device for steel structure bridge corrosion diseases in this application. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0049] In a first aspect, an embodiment of the present application provides a semantic segmentation method for corrosion damage in steel structure bridges, which is used to solve the problem of inaccurate segmentation caused by limited computing resources, too high a resolution of bridge pictures, and inability to run the program due to target recognition.

[0050] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of the semantic segmentation method for steel structure bridge corrosion damage in this application. Figure 1 As shown in Figure 1, the semantic segmentation method of steel structure bridge corrosion disease includes:

[0051] S1: Obtain high-resolution corrosion disease images of bridges and annotate them to generate annotated images. Based on the set cutting size and the set cutting step length, the corrosion disease images and the annotated images are cut into image blocks of set sizes.

[0052] Furthermore, in one embodiment, obtaining a high-resolution corrosion disease picture of a bridge and annotating it to generate an annotated image specifically includes:

[0053] S101: Obtain high-resolution pictures of bridge corrosion damage; for example, obtain high-resolution pictures of bridge corrosion damage such as 4K through drone photography or other means;

[0054] S102: Based on an image segmentation model or an active annotation method, the rust disease parts in the acquired rust disease picture are annotated to obtain an annotated image.

[0055] Specifically, the SAM semantic segmentation model and manual interaction can be used to achieve labeling, that is, a person clicks on the rusted area with a mouse to prompt the SAM model that there is rust there. The SAM model automatically marks the rusted area, and then a person manually determines whether the marked area is qualified. If it is unqualified, it is manually corrected to finally form a json format file.

[0056] Furthermore, in one embodiment, based on the set cutting size and the set cutting step length, the rust disease image and the annotated image are cut into image blocks of set sizes, specifically including:

[0057] S111: using a pixel block of a first size×a first size as a set cutting size, and using pixels of a second size as a set cutting step length; wherein a corresponding value of the second size is half of a corresponding value of the first size;

[0058] S112: Cut the rust disease image into a plurality of image blocks of first size×first size pixels, and cut the annotated image into a plurality of image blocks of first size×first size pixels in a cutting order from left to right and from top to bottom.

[0059] It should be noted that, in actual application, the value of the first size may be 1080, and the value of the second size may be 540. That is, with a pixel block of 1080×1080 as the set cutting size, and 540 pixels as the set cutting step, the rust disease image is cut into multiple image blocks of 1080×1080 pixels, and the annotated image is cut into multiple image blocks of 1080×1080 pixels in a cutting order from left to right and from top to bottom.

[0060] Furthermore, after the rust disease image and the annotated image are cut into image blocks of a set size, the method further includes:

[0061] Obtain image blocks of the annotated image in sequence, and determine whether the rust disease pixels in the obtained image blocks are greater than a preset percentage of the pixels of the entire image block:

[0062] If yes, keep the current image block of the annotated image;

[0063] If not, the current image block of the labeled image is deleted, and the image block corresponding to the rust disease picture is deleted at the same time.

[0064] Specifically, the image blocks of the annotated image are obtained in sequence, and then it is determined whether the target pixel (i.e., the pixel where the rust is located) in the currently obtained image block is greater than 5% of the pixels of the entire image block. If so, the currently obtained image block is retained; if not, the currently obtained image block is deleted, and the corresponding image block of the rust disease image is deleted.

[0065] S2: Use the improved three-branch network structure as the semantic segmentation algorithm, build a network model, and use the cut image blocks to train the network model;

[0066] Furthermore, in one embodiment, the improved three-branch network structure is used as a semantic segmentation algorithm to build a network model, and the network model is trained by cutting the image blocks, specifically including:

[0067] S201: Adding a pixel attention enhancement module to the I branch of PIDNet to obtain an improved three-branch network structure; PIDNet is a three-branch network architecture, which includes three branches: P (detailed), I (context), and D (proposing high-frequency features to predict boundary areas); by adding a pixel attention enhancement module to the I branch, it is used to retain local detail features and improve the accuracy of feature extraction;

[0068] S202: Use the improved three-branch network structure as a semantic segmentation algorithm to build a network model;

[0069] S203: The network model is trained using the cut rust disease pictures and the image blocks of the labeled images.

[0070] S3: Obtain the image to be predicted and cut the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and input the blocks into the trained network model to obtain a prediction result;

[0071] Further, in one embodiment, obtaining an image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the trained network model to obtain a prediction result, specifically includes:

[0072] S301: Acquire an image to be predicted, and cut the image to be predicted into a plurality of image blocks to be predicted of a first size×first size of pixels based on a set cutting size and a set cutting step size;

[0073] S302: Input the image blocks to be predicted into the trained network model, identify the rust disease parts in each image block to be predicted, and obtain the rust disease prediction result image of the image block to be predicted.

[0074] Specifically, according to the cutting method described in step S1, the image to be predicted is cut into multiple 1080×1080 pixel image blocks to be predicted, and then the cut image blocks to be predicted are input into the trained network model to obtain the rust disease prediction result image of the image blocks to be predicted.

[0075] S4: construct a blank image, and fill the rust disease prediction result image of each image block to be predicted into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results.

[0076] Furthermore, in one embodiment, a blank image is constructed, and the rust disease prediction result images of each image block to be predicted are filled into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results, specifically including:

[0077] S401: constructing a blank image with the same pixel size as the image to be predicted, and filling the blank image with pixels according to a first filling rule and a second filling rule in sequence to obtain a filled image;

[0078] S402: using the pixels of the third size as the moving step length, and sequentially filling the rust disease prediction result image into the filled image from left to right and from top to bottom; it should be noted that in the actual application process, the value of the third size may be 270;

[0079] S403: Crop the pixel-filled portion of the filled image to obtain a prediction result spliced ​​image;

[0080] The first filling rule is: dividing the number of pixels on the width of the image to be predicted by the first size to obtain a first remainder, and the difference between the first size and the first remainder is the number of pixels to be filled in the width direction of the blank image; dividing the number of pixels on the height of the image to be predicted by the first size to obtain a second remainder, and the difference between the first size and the second remainder is the number of pixels to be filled in the height direction of the blank image, and performing pixel filling on the blank image to obtain a pre-filled image;

[0081] The second filling rule is: fill the third size pixels around the pre-filled image to obtain a filled image, and the corresponding value of the third size is half of the corresponding value of the second size.

[0082] Specifically, construct a blank image, see Figure 2 As shown, Figure 2The pure white rectangle in the middle is the blank image. The pixel size of the blank image is consistent with the image to be predicted. Then the rust disease prediction result image needs to be filled into the blank image. In order to avoid serious splicing marks in the spliced ​​prediction result mosaic due to the overlap of the rust disease prediction result images (which is quite different from the original image to be predicted), the rust disease prediction result image is used to fill the central 540×540 pixel area in sequence. Therefore, it is necessary to fill the blank image with pixels, divide the number of pixels on the width of the image to be predicted by 1080, and get the first remainder a. The difference obtained by 1080-a is the blank image width square. The number of pixels that need to be filled upwards. Similarly, divide the number of pixels at the height of the image to be predicted by 1080 to get the second remainder b. The difference obtained by 1080-b is the number of pixels that need to be filled in the height direction of the blank image. After filling, a pre-filled image is obtained, and then 540 / 2=270 pixels are filled around the pre-filled image. In this way, a 1080×1080 pixel rust disease prediction result image can be used. A small block of 540×540 pixels in the center is filled in, and a moving step of 270 pixels is used for splicing. Then the pixel-filled part of the filled image is cropped to obtain a prediction result splicing image of the same size as the image to be predicted.

[0083] The semantic segmentation method for steel structure bridge corrosion damage in the embodiment of the present application effectively reduces the dependence on hardware equipment by cutting ultra-high resolution images into small images for training and reasoning, while also improving the recognition ability of small target areas. It can be well applied to UAV-based steel structure bridge corrosion damage detection projects.

[0084] In a second aspect, an embodiment of the present application also provides a semantic segmentation device for corrosion damage of steel structure bridges.

[0085] In one embodiment, referring to Figure 3 , Figure 3 This is a schematic diagram of the functional modules of the semantic segmentation device for steel structure bridge corrosion damage in this application. Figure 3 As shown, the semantic segmentation device for steel structure bridge corrosion disease includes: a labeling module, a training module, a prediction module, and a splicing module.

[0086] The annotation module is used to obtain high-resolution rust disease images of bridges and annotate them to generate annotated images. Based on the set cutting size and the set cutting step size, both the rust disease images and the annotated images are cut into image blocks of the set size; the training module is used to use the improved three-branch network structure as the semantic segmentation algorithm, build a network model, and train the network model using the image blocks obtained by cutting; the prediction module is used to obtain the image to be predicted and cut the image to be predicted into image blocks of the set size to be predicted based on the set cutting size and the set cutting step size, and input the trained network model to obtain the prediction result; the splicing module is used to construct a blank image, and fill the rust disease prediction result image of each image block to be predicted into the blank image based on the preset filling and splicing rules to complete the splicing of the prediction results.

[0087] Among them, the functional implementation of each module in the above-mentioned semantic segmentation device for rust damage of steel structure bridges corresponds to the steps in the above-mentioned semantic segmentation method for rust damage of steel structure bridges, and its functions and implementation processes will not be repeated here one by one.

[0088] In a third aspect, an embodiment of the present application provides a semantic segmentation device for steel structure bridge corrosion diseases. The semantic segmentation device for steel structure bridge corrosion diseases can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0089] Reference Figure 4 , Figure 4 The hardware structure diagram of the semantic segmentation device for steel structure bridge corrosion disease involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the semantic segmentation device for steel structure bridge corrosion disease may include a processor, a memory, a communication interface, and a communication bus.

[0090] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0091] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the semantic segmentation device for steel structure bridge corrosion damage, and the interface used to realize the interconnection of the semantic segmentation device for steel structure bridge corrosion damage with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0092] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0093] The processor may be a general-purpose processor, which may call the semantic segmentation program for steel structure bridge corrosion damage stored in the memory and execute the semantic segmentation method for steel structure bridge corrosion damage provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the semantic segmentation program for steel structure bridge corrosion damage is called may refer to the various embodiments of the semantic segmentation method for steel structure bridge corrosion damage of the present application, which will not be described in detail here.

[0094] Those skilled in the art will understand that Figure 4 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0095] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0096] The computer-readable storage medium of the present application stores a semantic segmentation program for steel structure bridge corrosion diseases, wherein when the semantic segmentation program for steel structure bridge corrosion diseases is executed by a processor, the steps of the semantic segmentation method for steel structure bridge corrosion diseases as described above are implemented.

[0097] Among them, the method implemented when the semantic segmentation program of steel structure bridge corrosion disease is executed can refer to the various embodiments of the semantic segmentation method of steel structure bridge corrosion disease in this application, and will not be repeated here.

[0098] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0099] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0100] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0101] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0102] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0104] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A semantic segmentation method for steel structure bridge corrosion damage, characterized in that: The steel structure bridge corrosion disease semantic segmentation method comprises: Obtain high-resolution corrosion disease images of bridges and annotate them to generate annotated images. Based on the set cutting size and the set cutting step length, cut both the corrosion disease images and the annotated images into image blocks of set sizes. The improved three-branch network structure is used as the semantic segmentation algorithm to build a network model, and the network model is trained by cutting the image blocks. Obtaining an image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the blocks into the trained network model to obtain a prediction result; Construct a blank image, and fill the rust disease prediction result image of each image block to be predicted into the blank image based on the preset filling and splicing rules to complete the splicing of the prediction results; The method of cutting the rust disease image and the annotated image into image blocks of a set size based on the set cutting size and the set cutting step size specifically includes: A pixel block of a first size×a first size is used as a set cutting size, and a pixel of a second size is used as a set cutting step length; According to the cutting order from left to right and from top to bottom, the rust disease image is cut into a plurality of image blocks of first size × first size pixels, and the labeled image is cut into a plurality of image blocks of first size × first size pixels; The second size corresponding value is half of the first size corresponding value; The step of obtaining the image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the trained network model to obtain the prediction result specifically includes: Acquire an image to be predicted, and cut the image to be predicted into a plurality of image blocks to be predicted of a first size×first size of pixels based on a set cutting size and a set cutting step size; Input the image blocks to be predicted into the trained network model, identify the rust disease parts in each image block to be predicted, and obtain the rust disease prediction result image of the image block to be predicted; The step of constructing a blank image and filling the rust disease prediction result image of each image block to be predicted into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results specifically includes: Constructing a blank image with the same pixel size as the image to be predicted, and filling the blank image with pixels according to the first filling rule and the second filling rule in sequence to obtain a filled image; Using the third largest pixel as the moving step, the rust disease prediction result image is filled into the filled image in order from left to right and from top to bottom; Crop the pixel-filled part of the padded image to obtain a spliced ​​image of the prediction result; The first filling rule is: dividing the number of pixels on the width of the image to be predicted by the first size to obtain a first remainder, and the difference between the first size and the first remainder is the number of pixels to be filled in the width direction of the blank image; dividing the number of pixels on the height of the image to be predicted by the first size to obtain a second remainder, and the difference between the first size and the second remainder is the number of pixels to be filled in the height direction of the blank image, and performing pixel filling on the blank image to obtain a pre-filled image; The second filling rule is: fill the third size pixels around the pre-filled image to obtain a filled image, and the corresponding value of the third size is half of the corresponding value of the second size.

2. A semantic segmentation method for corrosion damage of steel structure bridges as claimed in claim 1, characterized in that: The obtaining of high-resolution corrosion disease images of bridges and annotating them to generate annotated images specifically includes: Obtain high-resolution corrosion damage images of bridges; Based on the image segmentation model or the active annotation method, the rust disease parts in the acquired rust disease picture are annotated to obtain an annotated image.

3. The method for semantic segmentation of corrosion damage of steel structure bridges according to claim 1, characterized in that: After the rust disease images and the annotated images are cut into image blocks of a set size, the following steps are also included: Obtain image blocks of the annotated image in sequence, and determine whether the rust disease pixels in the obtained image blocks are greater than a preset percentage of the pixels of the entire image block: If yes, keep the current image block of the annotated image; If not, the current image block of the labeled image is deleted, and the image block corresponding to the rust disease picture is deleted at the same time.

4. The method for semantic segmentation of corrosion damage of steel structure bridges according to claim 1, characterized in that: The improved three-branch network structure is used as a semantic segmentation algorithm to build a network model, and the network model is trained by cutting the image blocks, which specifically includes: A pixel attention enhancement module is added to the I branch of PIDNet to obtain an improved three-branch network structure; Use the improved three-branch network structure as the semantic segmentation algorithm to build a network model; The network model is trained using the cut rust disease pictures and image blocks of the labeled images.

5. A semantic segmentation device for steel structure bridge corrosion damage, characterized in that: The steel structure bridge corrosion disease semantic segmentation device comprises: An annotation module is used to obtain high-resolution corrosion disease images of bridges and annotate them to generate annotated images. Based on a set cutting size and a set cutting step, the corrosion disease images and the annotated images are cut into image blocks of a set size. The training module is used to use the improved three-branch network structure as a semantic segmentation algorithm to build a network model and train the network model using image blocks obtained by cutting; A prediction module is used to obtain an image to be predicted and cut the image to be predicted into image blocks of a set size based on a set cutting size and a set cutting step size, and input the blocks into a trained network model to obtain a prediction result; A splicing module is used to construct a blank image and fill the rust disease prediction result image of each image block to be predicted into the blank image based on a preset filling splicing rule to complete the splicing of the prediction results; The method of cutting the rust disease image and the annotated image into image blocks of a set size based on the set cutting size and the set cutting step size specifically includes: A pixel block of a first size×a first size is used as a set cutting size, and a pixel of a second size is used as a set cutting step length; According to the cutting order from left to right and from top to bottom, the rust disease image is cut into a plurality of image blocks of first size × first size pixels, and the labeled image is cut into a plurality of image blocks of first size × first size pixels; The second size corresponding value is half of the first size corresponding value; The step of obtaining the image to be predicted and cutting the image to be predicted into image blocks to be predicted of a set size based on a set cutting size and a set cutting step size, and inputting the trained network model to obtain the prediction result specifically includes: Acquire an image to be predicted, and cut the image to be predicted into a plurality of image blocks to be predicted of a first size×first size of pixels based on a set cutting size and a set cutting step size; Input the image blocks to be predicted into the trained network model, identify the rust disease parts in each image block to be predicted, and obtain the rust disease prediction result image of the image block to be predicted; The step of constructing a blank image and filling the rust disease prediction result image of each image block to be predicted into the blank image based on a preset filling and splicing rule to complete the splicing of the prediction results specifically includes: Constructing a blank image with the same pixel size as the image to be predicted, and filling the blank image with pixels according to the first filling rule and the second filling rule in sequence to obtain a filled image; Using the third largest pixel as the moving step, the rust disease prediction result image is filled into the filled image in order from left to right and from top to bottom; Crop the pixel-filled part of the padded image to obtain a spliced ​​image of the prediction result; The first filling rule is: dividing the number of pixels on the width of the image to be predicted by the first size to obtain a first remainder, and the difference between the first size and the first remainder is the number of pixels to be filled in the width direction of the blank image; dividing the number of pixels on the height of the image to be predicted by the first size to obtain a second remainder, and the difference between the first size and the second remainder is the number of pixels to be filled in the height direction of the blank image, and performing pixel filling on the blank image to obtain a pre-filled image; The second filling rule is: fill the third size pixels around the pre-filled image to obtain a filled image, and the corresponding value of the third size is half of the corresponding value of the second size.

6. A semantic segmentation device for corrosion damage of steel structure bridges, characterized in that: The steel structure bridge corrosion disease semantic segmentation device includes a processor, a memory, and a steel structure bridge corrosion disease semantic segmentation program stored in the memory and executable by the processor. When the steel structure bridge corrosion disease semantic segmentation program is executed by the processor, the steps of the steel structure bridge corrosion disease semantic segmentation method as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a semantic segmentation program for steel structure bridge corrosion damage, wherein when the semantic segmentation program for steel structure bridge corrosion damage is executed by a processor, the steps of the semantic segmentation method for steel structure bridge corrosion damage as described in any one of claims 1 to 4 are implemented.

Images