Bridge structure outer surface defect identification method based on generative adversarial network

Abstract

The invention discloses a generative network-resisting bridge structure outer surface defect identification method, which comprises the following steps of: collecting an image of a bridge surface, cutting, rotating and zooming the image, marking the image to mark the position and the type of a defect, gradually reducing the size of a feature map by using a generator network to increase feature extraction, and identifying the defect of the outer surface of the bridge structure. Meanwhile, the discriminator network is used for gradually increasing the size of the feature map so as to increase feature recognition, the weights of the generator and the discriminator are initialized, the initial learning rate is set, the weights of the generator and the discriminator network are alternately updated, the quality of the image generated by the generator is evaluated, training is completed, and the generator is used for generating a bridge surface image; according to the method, a discriminator is used for judging whether the image contains defects or not, the defects and types are further determined on the discriminator, the problems in the adversarial generative network training process in the prior art can be effectively solved, and the accuracy of bridge structure outer surface defect recognition is improved.

Description

Technical Field

[0001] The invention relates to the field of information technology, and in particular to a method for identifying outer surface defects of a bridge structure using an adversarial generative network. Background Art

[0002] In the prior art, Generative Adversarial Networks (GANs) is a deep learning model proposed by Ian Goodfellow et al. in 2014. GANs consists of two neural networks: a generator and a discriminator. The two networks compete with each other. Through continuous training and learning, the generator can generate more and more realistic data, while the discriminator becomes more and more adept at distinguishing between real data and generated data.

[0003] The existing technology has the problem that the acquisition and preprocessing of bridge surface images are not comprehensive or accurate enough. At the same time, there is a problem of unstable training process, which leads to poor model performance. The model used performs poorly when processing bridge surface images, and cannot accurately identify defects or the defect identification accuracy is low. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the present invention provides a method for identifying outer surface defects of a bridge structure using an adversarial generative network.

[0005] The technical solution of the present invention is implemented as follows: a method for identifying defects on the outer surface of a bridge structure using an adversarial generative network, comprising:

[0006] S1. Collect images of the bridge surface, including normal state and state with defects;

[0007] S2, crop, rotate, and scale the image to standardize the image size and format;

[0008] S3, annotate the image and mark the location and type of the defect;

[0009] S4, use the generator network to gradually reduce the feature map size and increase feature extraction;

[0010] S5, use the discriminator network to gradually increase the size of the feature map and increase feature recognition;

[0011] S6, initialize the weights of the generator and discriminator, set the initial learning rate, and alternately update the weights of the generator and discriminator networks;

[0012] S7, evaluating the quality of the images generated by the generator, where the evaluation indicators include realism;

[0013] S8. After training is completed, the generator is used to generate an image of the bridge surface, and the discriminator is used to determine whether the image contains defects.

[0014] Furthermore, the normal state in step S1 includes:

[0015] No cracks: There are no cracks or only very small cracks on the bridge surface. These cracks are usually caused by natural shrinkage of the material and are not structural problems.

[0016] The protective layer is intact: the protective layer (asphalt, paint) on the bridge deck and structure surface is intact, without peeling or wear;

[0017] Structural integrity: The supporting structure, cables and towers of the bridge are not deformed or damaged;

[0018] Unobstructed drainage system: The bridge drainage system works normally without water accumulation or blockage;

[0019] Status with defects:

[0020] Cracks: Structural cracks, especially through-going cracks, indicate a loss of structural integrity;

[0021] Rust: Obvious rust appears on the metal parts, which affects the bearing capacity of the structure;

[0022] Peeling and abrasion: the protective layer peels off, exposing the base material;

[0023] Deformation: Any abnormal deformation of any part of the bridge, such as bending or twisting;

[0024] Displacement: displacement of bridge components, such as bearing displacement and bridge deck dislocation;

[0025] Water accumulation: Blockage in drainage systems leads to water accumulation, which accelerates the deterioration of structures.

[0026] Furthermore, the cropping, rotation and scaling in step S2 are as follows:

[0027] Set the image processing library to use OpenCV (Open Source Computer Vision Library) and PIL (Pillow Imaging Library);

[0028] Cropping: Use the cropping function in the image processing library to set the cropping area, which is determined by the cropping ratio and the starting coordinates. Apply the cropping formula: x_cropped = x*(1-crop_ratio), where x is the original image, crop_ratio is the cropping ratio, and x_cropped is the cropped image;

[0029] Rotation: Use the rotation function in the image processing library, set the rotation angle theta, and apply the rotation formula: x_rotated = x*cos(theta)-y*sin(theta), where x and y are the pixel coordinates of the original image, and x_rotated is the rotated image;

[0030] Scaling: Use the scaling function in the image processing library, set the scaling factor scale_factor, and apply the scaling formula: x_scaled = x*scale_factor, where x is the original image and x_scaled is the scaled image;

[0031] Standardize image size and format: adjust cropped, rotated, and scaled images to a uniform size and format, resize images to a specific resolution or number of pixels, adjust the image's pixel format, and convert from RGB to grayscale. The conversion process uses a simple average formula: Where R, G, B are the values ​​of the red, green, and blue channels respectively, and gray is the grayscale value.

[0032] Furthermore, in step S3, the image is annotated:

[0033] The calculation and annotation process describing the defect location and type:

[0034] The defect position is detected as (x_defect, y_defect) using the Canny edge detection algorithm. On the grayscale image, a mathematical formula is used to mark the defect position.

[0035] The width and height of the preset original image are W and H respectively;

[0036] The crop ratio is crop_ratio, and the starting coordinates are (start_x, start_y);

[0037] The defect location coordinates (x_defect, y_defect) are calculated using the following formula:

[0038] x_defect=\text{start_x}+\text{crop_ratio}\times W;

[0039] y_defect=\text{start_y}+\text{crop_ratio}\times H;

[0040] Among them, x_defect and y_defect: These two variables represent the horizontal (x-axis) and vertical (y-axis) coordinate positions of the detected defects in the image;

[0041] start_x and start_y: These two variables represent the starting coordinates of the clipping area, that is, the position where the clipping operation begins;

[0042] crop_ratio: This variable represents the crop ratio, which determines the size of the cropped area relative to the original image;

[0043] W and H: These two variables represent the width and height of the original image, that is, the size of the image in the horizontal and vertical directions.

[0044] Furthermore, the step S4 of reducing the feature map size is specifically as follows:

[0045] Residual Networks (ResNets) are used as the basic architecture of the generator network. ResNets solve the gradient vanishing and gradient exploding problems in deep network training by introducing residual blocks. At the same time, they can also extract features. The process of using ResNets as the generator network is as follows:

[0046] Input layer: The input of the generator network is a random noise vector, which consists of uniformly distributed vectors whose dimensions are associated with the size of the output feature map of the generator network;

[0047] Residual block: The residual block is the core structure of ResNet, which contains multiple convolutional layers and skip connections. Each residual block first extracts features through the convolutional layer, and then passes the input directly to the output through the skip connection to reduce the size of the feature map. The skip connection helps to retain the key information in the feature map and avoid the problem of gradient disappearance or explosion.

[0048] Downsampling layer: Add a downsampling layer between residual blocks, including an average pooling layer or a maximum pooling layer. The downsampling layer reduces the size of the feature map through pooling operations while retaining key features;

[0049] Upsampling layer: Add upsampling layers between residual blocks, including transposed convolution layers or interpolation layers. The upsampling layer increases the size of the feature map through interpolation operations while maintaining the integrity of the features;

[0050] Output layer: The output layer of the generator network is a convolutional layer that maps the feature map to the final output dimension, which is the output of the generator network. The convolution kernel of the output layer has the same number of channels and size as the size of the generated image.

[0051] Furthermore, the step S5 increases the size of the feature map, specifically:

[0052] The decoder network is used. The purpose of the decoder network is to start from low-dimensional features and gradually restore the size of the original image. The decoder network is used as the generator network process:

[0053] Input layer: The input to the generator network is a random noise vector, including a uniformly distributed vector whose dimension is related to the size of the output feature map of the generator network;

[0054] Upsampling layer: The decoder network contains multiple upsampling layers. Each upsampling layer increases the size of the input feature map and increases the feature extraction capability. Each upsampling layer usually uses a transposed convolution layer or an interpolation layer. The upsampling layer is followed by a convolution layer to extract features.

[0055] Activation function: Each convolutional layer is followed by a nonlinear activation function, including ReLU (Rectified Linear Unit);

[0056] Residual connection: Add residual connection between convolutional layers. Residual connection allows the network to pass input to output directly, thus avoiding the problem of gradient vanishing and gradient exploding.

[0057] Output layer: The output layer of the generator network is a convolutional layer that maps the feature map to the final output dimension, which is the output of the generator network. The convolution kernel of the output layer has the same number of channels and size as the size of the generated image.

[0058] Furthermore, in step S6, the weights of the generator and discriminator networks are updated alternately, specifically:

[0059] In the step S6, the weights of the generator and the discriminator are initialized, and the initial learning rate is set;

[0060] Initialize the weights of the generator and discriminator: The weights in the generator and discriminator need to be randomly initialized, using a small range of random numbers between [-0.1, 0.1];

[0061] Set the initial learning rate: The learning rate is a key parameter that determines the speed of gradient descent and network convergence;

[0062] Alternately update the weights of the generator and discriminator networks: first, fix the weights of the discriminator and use gradient descent to update the weights of the generator to minimize the discriminator's discrimination error for generated images. Then, fix the weights of the generator and use gradient descent to update the weights of the discriminator to maximize the discriminator's discrimination error for real images and generated images. Repeat this process until the predetermined number of iterations is reached or a certain stopping condition is met.

[0063] Furthermore, the image quality generated by the generator is evaluated in step S7, specifically:

[0064] In GANs, a discriminator network is used to evaluate the generated images, and the realism of the generated images is evaluated by calculating the discriminator's discrimination error on the image;

[0065] Image visual quality assessment uses the accuracy detection score of defects detected in the image.

[0066] Furthermore, the step S8 uses a discriminator to determine whether the image contains defects, and further determines the defects and types on the discriminator, specifically:

[0067] Generate bridge image: Use the trained generator network to generate bridge image. The bridge image generated by the generator network is preset to be a two-dimensional array I_gen.

[0068] Determine whether the image contains defects: Use the discriminator network to evaluate the generated bridge image. The output of the preset discriminator network is a probability value, which indicates the probability that the image is a real image.

[0069] If the output probability of the discriminator network is greater than the threshold value 0.5, the preset image is a real image, and the defect detection model is further used to check whether the image contains defects;

[0070] If the output probability of the discriminator network is less than or equal to the threshold value 0.5, the image is a generated image, and the image is assumed to contain no defects or to be verified using an additional dataset.

[0071] Beneficial Effects

[0072] The present invention can effectively solve the technical problems in the prior art and has the following beneficial effects:

[0073] Improve the accuracy of image acquisition and preprocessing: By using OpenCV and PIL image processing libraries, the bridge surface images are accurately cropped, rotated, and scaled to ensure the standardization of image size and format; the residual network and decoder network are used as the generator network, which can start from low-dimensional features, gradually restore to the size of the original image, and increase the feature extraction capability.

[0074] Enhance the stability and performance of the model: Use residual networks and decoder networks to solve the gradient vanishing and gradient exploding problems in deep network training, while being able to extract features; initialize the generator and discriminator weights, set the initial learning rate, and alternately update the weights of the generator and discriminator networks to improve the stability and performance of the model.

[0075] Improve the accuracy of defect recognition: Use the Canny edge detection algorithm to detect the defect location; Evaluate the quality of the images generated by the generator, including the fidelity to ensure the quality of the images generated by the generator.

[0076] In summary, the method provided by the present invention can effectively solve the technical problems in the prior art, improve the accuracy of identifying defects on the outer surface of bridge structures, and improve the reliability and safety of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a structural block diagram of a bridge structure outer surface defect recognition method using an adversarial generation network in an embodiment of the present invention;

[0078] Figure 2 This is a flowchart of a method for identifying defects on the outer surface of a bridge structure using an adversarial generative network in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0080] The preferred implementation methods of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these implementation methods are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0081] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.

[0082] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" and the like specify the presence of stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0083] See also Figure 1-2 As shown, a bridge structure outer surface defect recognition method based on adversarial generative network includes:

[0084] S1. Collect images of the bridge surface, including normal state and state with defects;

[0085] S2, crop, rotate, and scale the image to standardize the image size and format;

[0086] S3, annotate the image and mark the location and type of the defect;

[0087] S4, use the generator network to gradually reduce the feature map size and increase feature extraction;

[0088] S5, use the discriminator network to gradually increase the size of the feature map and increase feature recognition;

[0089] S6, initialize the weights of the generator and discriminator, set the initial learning rate, and alternately update the weights of the generator and discriminator networks;

[0090] S7, evaluating the quality of the images generated by the generator, where the evaluation indicators include fidelity and accuracy of defect detection;

[0091] S8. After training is completed, the generator is used to generate an image of the bridge surface, and the discriminator is used to determine whether the image contains defects, and the defects and types are further determined on the discriminator.

[0092] Furthermore, the normal state and the defective state in the S1 step are converted into numerical codes, and the following coding rules are preset: no cracks: 0, intact protective layer: 1, complete structure: 2, unobstructed drainage system: 3, cracks: 4, rust: 5, peeling and wear: 6, deformation: 7, displacement: 8, water accumulation: 9; for each bridge, according to its state, a state coding vector V is preset, V = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].

[0093] Furthermore, in step S2, the state data in step S1 is encoded, and combined with the mathematical calculation method in step S2, the original image is converted into a preset format:

[0094] Image Processing:

[0095] The preset image is a two-dimensional array I, where I[x,y] represents the pixel value of the image at position (x,y);

[0096] Crop:

[0097] The crop ratio crop_ratio is a value between 0 and 1;

[0098] Calculate the cropped image size:

[0099] height_cropped=height*(1-crop_ratio);

[0100] width_cropped=width*(1-crop_ratio);

[0101] Rotation:

[0102] The rotation angle theta is an angle value, indicating the angle of clockwise rotation;

[0103] For each pixel (x,y) in the image, calculate the rotated coordinates (x_rotated,y_rotated):

[0104] x_rotated=x*cos(theta)-y*sin(theta);

[0105] y_rotated=x*sin(theta)+y*cos(theta);

[0106] Apply the rotation formula: I_rotated[x_rotated,y_rotated]=I[x,y];

[0107] Zoom:

[0108] The scaling factor scale_factor is a value greater than 0;

[0109] Calculate the scaled image size:

[0110] height_scaled=height*scale_factor;

[0111] width_scaled=width*scale_factor;

[0112] Apply the scaling formula: I_scaled[x_scaled,y_scaled]=I[x / scale_factor,y / scale_factor];

[0113] Standardize image sizes and formats:

[0114] Resize the image to uniform dimensions, using the default values ​​new_height and new_width:

[0115] I_resized[new_x,new_y]=I_scaled[new_x*width_scaled / new_width,new_y*height_scaled / new_height];

[0116] Convert the image from RGB to grayscale: gray[new_x,new_y]=(R[new_x,new_y]+G[new_x,new_y]+B[new_x,new_y]) / 3;

[0117] Where R, G, B are the values ​​of the red, green and blue channels respectively, and gray is the grayscale value;

[0118] Through the above steps, the state data in step S1 is encoded, and combined with the mathematical calculation method in step S2, the original image is converted into a preset format, wherein the physical meaning of the characters in the calculation formula in the above content is:

[0119] Data encoding:

[0120] V: state encoding vector, representing different states of the bridge;

[0121] 0,1,...,9: state-encoded values, each value corresponds to a specific bridge state; Image processing:

[0122] Crop:

[0123] Height: The height of the original image (in pixels);

[0124] Width: width of the original image (number of pixels);

[0125] crop_ratio: crop ratio, which means the ratio of the cropped area to the original image; height_cropped: the height of the cropped image;

[0126] width_cropped: the width of the cropped image;

[0127] I: pixel matrix of the original image;

[0128] I_cropped: cropped image pixel matrix;

[0129] x, y: pixel coordinates in the original image;

[0130] I_cropped[x,y]: the pixel value at position (x,y) in the cropped image;

[0131] Rotation:

[0132] theta: rotation angle, indicating the angle of the image rotated clockwise (in radians or degrees); x_rotated, y_rotated: the new coordinates of the rotated pixel;

[0133] cos(theta): cosine of the rotation angle theta;

[0134] sin(theta): sine value of rotation angle theta;

[0135] I_rotated: rotated image pixel matrix;

[0136] Zoom:

[0137] scale_factor: scaling factor, indicating the ratio of image enlargement or reduction;

[0138] height_scaled: the height of the scaled image;

[0139] width_scaled: the width of the scaled image;

[0140] I_scaled: scaled image pixel matrix;

[0141] x_scaled, y_scaled: scaled pixel coordinates;

[0142] Standardize image sizes and formats:

[0143] new_height: the new height of the adjusted image;

[0144] new_width: the new width of the adjusted image;

[0145] I_resized: the pixel matrix of the resized image;

[0146] new_x, new_y: pixel coordinates after resizing;

[0147] R, G, B: represent the pixel values ​​of the red, green and blue channels of the image respectively;

[0148] gray: The pixel value of a grayscale image represents the grayscale level of the image.

[0149] Furthermore, based on the completion of steps S1-S2 in step S3, step S3 involves annotating the image to describe the defect location and type. The following is a detailed process and required calculations for completing this step:

[0150] Defect Detection:

[0151] Use the Canny edge detection algorithm to detect the location of defects on the grayscale image. The Canny algorithm will return a set of edge points from which the location of the defect can be determined;

[0152] Calculate the defect position coordinates: According to the edge points detected by the Canny algorithm, determine the defect position coordinates (x_edge, y_edge) in the cropped image;

[0153] Convert the cropped defect position coordinates to the original image coordinates: In order to convert the cropped defect position coordinates to the original image coordinates, it is necessary to consider the starting coordinates (start_x, start_y) of the cropping operation and the crop ratio crop_ratio. The following is the conversion formula:

[0154] x_defect=start_x+crop_ratio*W;

[0155] y_defect=start_y+crop_ratio*H;

[0156] Where: x_defect and y_defect are the horizontal (x-axis) and vertical (y-axis) coordinate positions of the defect in the original image;

[0157] start_x and start_y are the starting coordinates of the clipping area, that is, the position where the clipping operation starts;

[0158] crop_ratio is the crop ratio, which determines the size of the cropped area relative to the original image;

[0159] W and H are the width and height of the original image;

[0160] Mark defects: Use the defect location coordinates (x_defect, y_defect) calculated above to mark the location of the defect on the original image. This is achieved by drawing a mark (circle or rectangular box) on the original image. The following is a description of the marking process:

[0161] for each(x_edge,y_edge)in edge_points:

[0162] x_defect=start_x+crop_ratio*W*(x_edge / width_cropped);

[0163] y_defect=start_y+crop_ratio*H*(y_edge / height_cropped);

[0164] draw_marker(I,(x_defect,y_defect),marker_size,arker_color);

[0165] Where: edge_points is the set of edge points detected by the Canny algorithm;

[0166] width_cropped and height_cropped are the width and height of the cropped image;

[0167] draw_marker is a function that draws a marker at position (x_defect, y_defect) on image I;

[0168] marker_size and marker_color are the size and color of the marker;

[0169] Through this process, the detected defect locations are accurately marked on the original image;

[0170] In the above steps, the calculation results x_defect and y_defect represent the exact position coordinates of the defects detected in the original image. These coordinates do not directly correspond to a specific defect type, but indicate the location of the defect in the image. In order to determine the defect type corresponding to these coordinates, it is necessary to combine the defect type defined in step S1 and the image analysis performed in step S3;

[0171] According to the defect types defined in step S1, the following are the corresponding defects:

[0172] No crack: If the detected defect location does not have features that match the "no crack" state, it is not a crack defect;

[0173] Crack: If the detected defect location matches the "crack" state, that is, the characteristics of a structural crack are observed in the image, then x_defect and y_defect correspond to the location of the crack defect;

[0174] Rust: If the detected defect location matches the "rust" state, that is, the characteristics of rust on the metal part are observed in the image, then x_defect and y_defect correspond to the location of the rust defect;

[0175] Peeling and Wear: If the detected defect location matches the "Peeling and Wear" state, that is, the characteristics of protective layer peeling are observed in the image, then x_defect and y_defect correspond to the location of the peeling and wear defects;

[0176] Deformation: If the detected defect location matches the "deformation" state, that is, abnormal deformation of the bridge part is observed in the image, then x_defect and y_defect correspond to the location of the deformation defect;

[0177] Displacement: If the detected defect location matches the "displacement" state, i.e. the displacement of the bridge component is observed in the image, then x_defect and y_defect correspond to the location of the displacement defect;

[0178] Water accumulation: If the detected defect location matches the "water accumulation" state, that is, water accumulation caused by a clogged drainage system is observed in the image, then x_defect and y_defect correspond to the location of the water accumulation defect.

[0179] Furthermore, in the S4-S5 steps, in order to reduce or increase the size of the feature map, in the S1-S3 steps, the position of the defect is detected by the Canny edge detection algorithm, and the position coordinates of the defect are calculated, and these coordinates are used as inputs of the generator network to generate images similar to the real images. At the same time, these coordinates are also used as inputs of the discriminator network to help the discriminator network distinguish between the generated images and the real images. Through the S4-S5 steps, the size of the feature map is gradually reduced and increased to enhance the capabilities of feature extraction and feature recognition, which helps the generator network and the discriminator network to better identify and generate images similar to the real images.

[0180] Specifically, the reduction or increase of feature map size is further processed using a generator network and a discriminator network:

[0181] Reduce feature map size (generator network):

[0182] Input data: The coordinates of the defect positions detected by Canny edges are used as the input of the generator network. The coordinates are converted into feature maps. The size of the feature map determines the input size of the generator network.

[0183] Generator network architecture: The generator network contains multiple convolutional layers and upsampling layers to gradually reduce the size of the feature map. The convolutional layer is used to extract features, and the upsampling layer is used to increase the resolution of the feature map.

[0184] Output: The output of the generator network is a low-resolution feature map that contains the key features in the original image.

[0185] Increase the feature map size (discriminator network):

[0186] Input data: The output of the generator network is used as the input of the discriminator. The input of the discriminator network is a low-resolution feature map, which needs to be enlarged to the size of the original image.

[0187] Discriminator network architecture: The discriminator network consists of multiple convolutional layers and downsampling layers, which are used to gradually increase the size of the feature map. The convolutional layer is used to extract features, and the downsampling layer is used to reduce the resolution of the feature map.

[0188] Output: The output of the discriminator network is a high-resolution feature map that contains the key features identified by the discriminator network.

[0189] Among them, the preset data explains the calculation process:

[0190] In reducing feature map size (generator network):

[0191] Input data:

[0192] A preset defect location coordinate data set is preset, which contains 100 defect location coordinates, and each coordinate is represented by a vector, [x1, y1, x2, y2, …, x100, y100];

[0193] Generator network architecture:

[0194] The generator network contains two convolutional layers and two upsampling layers;

[0195] The first convolution layer uses a 3x3 convolution kernel, a stride of 1, and the same padding, and the output feature map size is 64x64;

[0196] The second convolutional layer uses a 3x3 convolution kernel, a stride of 1, and the same padding, and the output feature map size is 32x32;

[0197] The first upsampling layer uses a 2x2 transposed convolution kernel with a stride of 2 and an output feature map size of 64x64;

[0198] The second upsampling layer uses a 2x2 transposed convolution kernel with a stride of 2 and an output feature map size of 128x128;

[0199] Output:

[0200] The output of the generator network is a low-resolution feature map of size 128x128;

[0201] Among them, same is one of the ways to process the edges of the input feature map. "Same" padding: also called "validpadding" or "zero padding". In this case, the edges of the input feature map will not be copied, but directly covered by the convolution kernel. The size of the output feature map of the convolution layer will be the same as the size of the input feature map.

[0202] In increasing the feature map size (discriminator network):

[0203] Input data:

[0204] The output of the generator network is a low-resolution feature map of size 128x128;

[0205] Discriminator network architecture:

[0206] The discriminator network contains two convolutional layers and two downsampling layers;

[0207] The first convolution layer uses a 3x3 convolution kernel, a stride of 1, and the same padding, and the output feature map size is 128x128;

[0208] The second convolution layer uses a 3x3 convolution kernel, a stride of 1, the same padding, and an output feature map size of 64x64;

[0209] The first downsampling layer uses a 2x2 maximum pooling layer with a stride of 2 and an output feature map size of 64x64;

[0210] The second downsampling layer uses a 2x2 maximum pooling layer with a stride of 2 and an output feature map size of 32x32;

[0211] Output:

[0212] The output of the discriminator network is a high-resolution feature map of size 32x32.

[0213] Furthermore, in step S6, the weights of the generator and the discriminator are initialized, the initial learning rate is set, and the weights of the generator and the discriminator networks are updated alternately, specifically:

[0214] 601. Initialize the weights of the generator and the discriminator, select a preset initial learning rate η, and use a random number generator to initialize the weights Wgen and Wdisc of the generator network G and the discriminator network D;

[0215] 602. Set the initial learning rate, the preset initial learning rate η, η = 0.001;

[0216] 603. Alternately train the generator and discriminator networks. In each iteration, first train the discriminator network and then train the generator network.

[0217] Train the discriminator network:

[0218] Randomly select a batch of real images and generated images from the dataset;

[0219] The input to the discriminator network is a combination of real and generated images;

[0220] The output of the discriminator network is the probability of real images and generated images;

[0221] Calculate the loss of the discriminator network, which is the difference between the probability of real images and the probability of generated images;

[0222] Update the weight Wdisc of the discriminator network, the formula is expressed as:

[0223] ΔW disc =-η·gradient_discriminator

[0224] W disc =W disc +ΔW disc

[0225] Train the generator network:

[0226] Generate a batch of random noise data as input to the generator;

[0227] The output of the generator network is the generated image;

[0228] Calculate the loss of the generator network as the difference between the probability of the generated image and the probability of the real image;

[0229] Update the weights W of the generator network gen , the formula is:

[0230] ΔW disc =-η·gradient_discriminator

[0231] W gen =W gen +ΔW gen

[0232] The above steps complete the initialization of the generator and discriminator weights, set the initial learning rate, and alternately update the weights of the generator and discriminator networks, where W gen is the generator network, W disc is the discriminator network.

[0233] Furthermore, the image quality generated by the generator is evaluated in step S7, specifically:

[0234] Fidelity Assessment:

[0235] The quality of the images generated by the preset generator network is represented by a realism score based on the output probability of the discriminator network;

[0236] The realism score is defined as a value between 0 and 1, where 1 means the generated image is completely realistic and 0 means the generated image is completely fake. The formula for calculating the realism score is:

[0237]

[0238] Defect detection accuracy evaluation:

[0239] The accuracy of defect detection in the images generated by the preset generator network is represented by a defect detection score;

[0240] The defect detection score is calculated as:

[0241]

[0242] Furthermore, in step S8, a generator is used to generate a bridge image, and a discriminator is used to determine whether the image contains defects, specifically:

[0243] Generate bridge image: Use the trained generator network to generate bridge image. The bridge image generated by the generator network is preset to be a two-dimensional array I_gen.

[0244] Determine whether the image contains defects: Use the discriminator network to evaluate the generated bridge image. The output of the preset discriminator network is a probability value, which indicates the probability that the image is a real image.

[0245] If the output probability of the discriminator network is greater than the threshold value 0.5, the preset image is a real image, and the defect detection model is further used to check whether the image contains defects;

[0246] If the output probability of the discriminator network is less than or equal to the threshold value 0.5, the image is a generated image, and the image is assumed to contain no defects or to be verified using an additional dataset.

[0247] Through the above steps, the quality of the image generated by the generator network is evaluated, and the discriminator network and defect detection model are used to determine whether the image contains defects, and the bridge defect types mentioned in steps S1-S3 are matched.

[0248] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0249] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and rules of the present invention shall be included in the protection scope of the present invention.

Claims

1. A bridge structure outer surface defect recognition method based on adversarial generative network, characterized by: S1. Collect images of the bridge surface, including normal state and state with defects; S2, crop, rotate, and scale the image to standardize the image size and format; S3, annotate the image and mark the location and type of the defect; S4, use the generator network to gradually reduce the feature map size and increase feature extraction; S5, use the discriminator network to gradually increase the size of the feature map and increase feature recognition; S6, initialize the weights of the generator and discriminator, set the initial learning rate, and alternately update the weights of the generator and discriminator networks; S7, evaluating the quality of the images generated by the generator, where the evaluation indicators include realism; S8. After training is completed, the generator is used to generate an image of the bridge surface, and the discriminator is used to determine whether the image contains defects.

2. According to claim 1, a bridge structure outer surface defect recognition method based on adversarial generative network is characterized by: The S2 step performs cropping, rotation, and scaling as follows: Apply the cropping formula: x_cropped = x*(1-crop_ratio), where x is the original image, crop_ratio is the cropping ratio, and x_cropped is the cropped image; Apply the rotation formula: x_rotated = x*cos(theta)-y*sin(theta), where x and y are the pixel coordinates of the original image and x_rotated is the rotated image; Apply the scaling formula: x_scaled = x * scale_factor, where x is the original image and x_scaled is the scaled image; Adjust the pixel format of the image and convert it from RGB to grayscale. The conversion process uses a simple average formula: Where R, G, B are the values ​​of the red, green, and blue channels respectively, and gray is the grayscale value.

3. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: In the step S3, the image is annotated and the defect position is detected using the Canny edge detection algorithm, which is calculated using the following formula: x_defect=\text{start_x}+\text{crop_ratio}\times W; y_defect=\text{start_y}+\text{crop_ratio}\times H; Among them, x_defect and y_defect: These two variables represent the horizontal x-axis and vertical y-axis coordinate positions of the detected defects in the image; start_x and start_y: These two variables represent the starting coordinates of the clipping area, that is, the position where the clipping operation begins; crop_ratio: This variable represents the crop ratio, which determines the size of the cropped area relative to the original image; W and H: These two variables represent the width and height of the original image, that is, the size of the image in the horizontal and vertical directions.

4. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: In the S4 step, a residual network is used as the basic architecture of the generator network, and ResNets is used as the generator network.

5. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: In the S5 step, a decoder network is adopted and the decoder network is used as a generator network.

6. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: In the step S6, the weights in the generator and the discriminator need to be randomly initialized using random numbers in a small range between [-0.1, 0.1]. First, the weight of the discriminator is fixed, and the weight of the generator is updated using gradient descent to minimize the discrimination error of the discriminator for the generated image. Then, the weight of the generator is fixed, and the weight of the discriminator is updated using gradient descent to maximize the discrimination error of the discriminator for the real image and the generated image. This process is repeated until the predetermined number of iterations is reached.

7. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: In the step S7, GANs is used to evaluate the quality of the image generated by the generator. In GANs, a discriminator network is used to evaluate the generated image. The realism of the generated image is evaluated by calculating the discrimination error of the discriminator on the image. The image visual quality is evaluated using the accuracy detection score of defect detection in the image.

8. The bridge structure outer surface defect recognition method based on adversarial generative network according to claim 1 is characterized by: The step S8 uses a discriminator to determine whether the image contains defects, and further determines the defects and types based on the discriminator: Generate a bridge image using the trained generator network. The bridge image generated by the generator network is a two-dimensional array I_gen. If the output probability of the discriminator network is greater than the threshold value 0.5, the preset image is a real image, and the defect detection model is further used to check whether the image contains defects; If the output probability of the discriminator network is less than or equal to the threshold value 0.5, the image is a generated image, and the image is assumed to contain no defects or to be verified using an additional dataset.

Images