A bridge disease image texture feature extraction and classification method and system

Abstract

The application discloses a bridge disease image texture feature extraction and classification method and system, and particularly relates to the technical field of image feature extraction and classification, and is used for solving the problem that the existing rectangular detection frame does not fit the physical boundary of the disease, thereby causing non-stationary deviation of the feature vector; the texture contrast is calculated pixel by pixel from each boundary to the inside in the disease rectangular candidate frame, the demarcation line is determined according to the texture contrast change, and the core disease area and the disease transition zone area are divided, the texture contrast difference sequence on both sides along the demarcation line is calculated point by point, and the texture virtuality is determined according to the local entropy of the difference sequence, the texture feature vector of the transition zone area is weighted, the vector difference degree is calculated with the texture feature vector of the core disease area, when the difference degree exceeds the threshold value, the texture feature vector of the core disease area is corrected by weighting, the corrected texture feature vector is input into a texture classifier, the disease category confidence is corrected according to the classification result, and the final classification result is output.

Description

Technical Field

[0001] This invention relates to the field of image feature extraction and classification technology, and more specifically, to a method and system for extracting and classifying texture features from bridge damage images. Background Technology

[0002] In the automated inspection of surface defects on bridge concrete, object detection models based on convolutional neural networks and texture feature extraction algorithms based on gray-level co-occurrence matrices are often used in combination. The typical process is as follows: first, the object detection model locates the defect area in the image and outputs a rectangular candidate box; then, the gray-level co-occurrence matrix is ​​used to calculate the texture statistics of the area within the box; finally, based on the extracted texture feature vectors, the defect types such as cracks, spalling, and honeycomb pitting are classified, which to a certain extent automates the extraction and classification of defect features.

[0003] However, the rectangular detection boxes output by the target detection model are difficult to precisely fit the actual physical boundaries of the disease, especially for diseases with irregular or gradually changing edges. The box will inevitably contain the disease area, the healthy background area, and the transition area between the two. Since the gray-level co-occurrence matrix performs global statistics on all pixels in the box, the intermediate state presented by the transition area, which is neither typical disease texture nor typical background texture, will be included as effective information in the feature vector. The inclusion of this non-stationary texture component will cause uncontrollable feature shifts in the feature space for the same type of disease due to the different proportions of its edges covered by the detection box. This will amplify the intra-class divergence and thus impair the accuracy and stability of subsequent classification processes. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method and system for extracting and classifying texture features of bridge defects to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for extracting and classifying texture features from bridge defect images, comprising: S1: Acquire bridge surface images, use convolutional neural networks to detect defects, and output candidate rectangular boxes for defects and confidence scores for defect categories; S2: Calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel within the candidate bounding box of the disease rectangle from each boundary inward. Determine the boundary line between the disease and the transition zone based on the change in texture contrast. Take the area inside the boundary line as the core disease area and the remaining area as the disease transition zone area. S3: Extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; S4: Calculate the difference in texture contrast between the core disease area and the disease transition zone point by point along the boundary line to obtain the difference sequence. Calculate the local entropy of the difference sequence to determine the texture fidelity. Use the texture fidelity to weight the second texture feature vector to obtain the weighted second texture feature vector. S5: Calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is reduced in weight to obtain the corrected texture feature vector. S6: Input the modified texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by modifying the confidence of the disease category.

[0006] Furthermore, S1 includes: Obtain bridge surface images and input them into a convolutional neural network; Convolutional neural networks perform convolution and pooling processing on bridge surface images to output feature maps; Candidate regions are extracted from the feature map to generate multiple candidate regions; For each candidate region, perform bounding box regression and disease category confidence prediction; From multiple candidate regions, select candidate regions whose disease category confidence is higher than a preset confidence threshold as disease rectangular candidate boxes; Output disease candidate rectangles and the disease category confidence score for each disease candidate rectangle.

[0007] Furthermore, S2 includes: Within the candidate bounding box of the disease rectangle, the gray-level co-occurrence matrix is ​​calculated pixel by pixel from the four boundaries of the candidate bounding box into the inside of the candidate bounding box, and the texture contrast of each pixel line is extracted. Along each boundary inwards, the texture contrast of each pixel line is arranged sequentially to obtain the texture contrast change sequence corresponding to each boundary; The locations where abrupt changes in texture contrast are determined from the texture contrast change sequence, and the pixel lines corresponding to the locations of abrupt changes are used as boundary lines. The area enclosed by the four boundary lines is designated as the core disease area, and the area between the boundary lines and the boundary of the disease rectangle candidate box is designated as the disease transition zone area.

[0008] Further, determining the location where abrupt changes in texture contrast occur from the texture contrast change sequence includes: subtracting the texture contrast value of the previous pixel line from the texture contrast value of each pixel line in the texture contrast change sequence to obtain a texture contrast difference value; comparing the texture contrast difference value with a preset abrupt change threshold; and taking the pixel line corresponding to the location where the texture contrast difference value first exceeds the preset abrupt change threshold as the location where abrupt changes in texture contrast occur.

[0009] Furthermore, S3 includes: Calculate the gray-level co-occurrence matrix for the core disease area, and extract texture contrast, texture energy, texture entropy and texture correlation from the gray-level co-occurrence matrix to form the first texture feature vector; The gray-level co-occurrence matrix is ​​calculated for the disease transition zone region. Texture contrast, texture energy, texture entropy and texture correlation are extracted from the gray-level co-occurrence matrix to form the second texture feature vector.

[0010] Furthermore, S4 includes: Extract the texture contrast values ​​of the core disease area and the texture contrast value of the disease transition zone area by sampling point along the boundary line; Calculate the absolute value of the difference between the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area, and arrange the absolute values ​​in the order of the sampling points of the boundary line to obtain the difference sequence. Calculate the local entropy of the difference sequence and compare the local entropy with a preset entropy threshold. If the local entropy is lower than the preset entropy threshold, the texture fidelity is determined to be high; if the local entropy is not lower than the preset entropy threshold, the texture fidelity is determined to be low. The second texture feature vector is obtained by performing a scalar multiplication operation on the texture realism.

[0011] Furthermore, S5 includes: Calculate the cosine distance between the first texture feature vector and the weighted second texture feature vector, and use the cosine distance as the vector difference. Compare the vector dissimilarity to the dissimilarity threshold; If the vector difference exceeds the difference threshold, a weighting coefficient is calculated based on the vector difference, and the weighting coefficient is multiplied by the first texture feature vector to obtain the corrected texture feature vector. If the vector difference does not exceed the difference threshold, then the first texture feature vector is used as the corrected texture feature vector.

[0012] Furthermore, calculating the weighting coefficient based on the vector dissimilarity includes: inputting the vector dissimilarity into a weighting function, which is a non-linear function that monotonically decreases the vector dissimilarity from the dissimilarity threshold to the dissimilarity upper limit; and outputting the weighting coefficient from the weighting function, with the weighting coefficient ranging from zero to one.

[0013] Furthermore, S6 includes: The modified texture feature vector is input into the texture classifier. The texture classifier calculates the similarity between the modified texture feature vector and the pre-stored disease texture category template. The disease texture category corresponding to the pre-stored disease texture category template with the highest similarity is taken as the output. Compare whether the disease texture category and the disease category corresponding to the disease category confidence level are consistent; If the disease texture category matches the disease category corresponding to the disease category confidence score, then increase the disease category confidence score as the final confidence score; If the disease texture category is inconsistent with the disease category corresponding to the disease category confidence score, then the disease texture category will replace the disease category corresponding to the disease category confidence score, and the confidence score of the replaced disease category will be used as the final confidence score. Output the final classification result, which includes the disease texture category and the final confidence score.

[0014] On the other hand, the present invention provides a bridge defect image texture feature extraction and classification system, comprising: The detection module is used to acquire images of the bridge surface, detect defects using a convolutional neural network, and output candidate rectangular boxes for defects and confidence scores for defect categories. The traceability and segmentation module is used to calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel within the candidate bounding box of the disease rectangle from each boundary inward. Based on the change in texture contrast, the boundary line between the disease and the transition zone is determined. The area inside the boundary line is taken as the core disease area, and the remaining area is taken as the disease transition zone area. The feature extraction module is used to extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; The virtual-real weighting module is used to calculate the difference in texture contrast between the core disease area and the disease transition zone area point by point along the boundary line to obtain the difference sequence, calculate the local entropy of the difference sequence to determine the virtual-real degree of texture, and weight the second texture feature vector with the virtual-real degree of texture to obtain the weighted second texture feature vector; The weight reduction correction module is used to calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is weighted and corrected to obtain the corrected texture feature vector. The classification output module is used to input the corrected texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by correcting the confidence of the disease category.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By calculating the texture contrast pixel by pixel from each boundary inward and determining the boundary line based on the change in texture contrast, the candidate bounding box of the disease rectangle is automatically divided into the core disease area and the disease transition zone area. This separates the transition zone texture that was originally mixed into the global statistics, avoiding the contamination of disease features by atypical textures. The difference in texture contrast between the two sides is calculated point by point along the boundary line to form a difference sequence. The local entropy of the difference sequence is used to measure the degree of order of the difference along the boundary. After determining the virtuality of the texture using this degree of order, the feature vector of the transition zone texture is weighted, so that the real disease extension texture and the box cutting pseudo texture are differentiated at the feature level, suppressing the non-stationary perturbation of the feature vector by the transition zone texture from the source.

[0016] 2. The vector difference is calculated between the weighted transition zone texture feature vector and the core disease texture feature vector. When the difference exceeds the threshold, the core disease texture feature vector is reduced in weight to form a cascade correction mechanism. This mechanism ensures that the feature vector received by the classifier retains the true texture information of the core area while eliminating the interference components introduced by the box edge error. The distribution of the same type of disease in the feature space is more concentrated, and the inter-class boundaries between different diseases are clearer, ultimately improving the accuracy and stability of bridge surface disease texture classification. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for extracting and classifying texture features from bridge defects images according to the present invention; Figure 2 This is a schematic diagram of the structure of a bridge defect image texture feature extraction and classification system according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Figure 1 This invention presents a method for extracting and classifying texture features from bridge defect images, comprising: S1: Acquire bridge surface images, use convolutional neural networks to detect defects, and output candidate rectangular boxes for defects and confidence scores for defect categories; S2: Calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel within the candidate bounding box of the disease rectangle from each boundary inward. Determine the boundary line between the disease and the transition zone based on the change in texture contrast. Take the area inside the boundary line as the core disease area and the remaining area as the disease transition zone area. S3: Extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; S4: Calculate the difference in texture contrast between the core disease area and the disease transition zone point by point along the boundary line to obtain the difference sequence. Calculate the local entropy of the difference sequence to determine the texture fidelity. Use the texture fidelity to weight the second texture feature vector to obtain the weighted second texture feature vector. S5: Calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is reduced in weight to obtain the corrected texture feature vector. S6: Input the modified texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by modifying the confidence of the disease category.

[0020] In the specific implementation of S1, a bridge surface image is acquired and input into a convolutional neural network. The bridge surface image is a digital image obtained by a visible light camera mounted on a bridge inspection platform, which can be an industrial camera mounted on a bridge inspection vehicle, an aerial camera mounted on a drone, or a camera module in a handheld inspection device. The bridge surface image is input into the convolutional neural network in the form of a digital image matrix, where each element corresponds to the grayscale value of a pixel. When the bridge surface image is a color image, it is first converted to a grayscale image. The conversion method involves merging the red, green, and blue channel values ​​of each pixel in the color image into a single grayscale value using a weighted average. The weight of the red channel value is 0.299, the weight of the green channel value is 0.587, and the weight of the blue channel value is 0.114. The grayscale image's digital image matrix is ​​then input into the convolutional neural network.

[0021] Convolutional Neural Networks (CNNs) perform convolution and pooling processing on bridge surface images, outputting feature maps. A CNN consists of alternating convolutional and pooling layers. Each convolutional layer contains several convolutional kernels, and each kernel is a weight matrix. Each weight value in the weight matrix is ​​a floating-point number updated via backpropagation during CNN training. During the convolution process, the kernel slides across the input image data with a preset stride, moving along the row and column directions. At each position, the kernel's weight matrix is ​​multiplied and summed with the corresponding elements of the input image data covered by the kernel, resulting in a response value. After traversing the entire input image data, the kernel forms a response map. All kernels within the same convolutional layer output their own response maps, which are then stacked along the depth direction to form the intermediate feature map output by that convolutional layer. The pooling layer downsamples the intermediate feature map output from the previous convolutional layer. This downsampling is achieved by either merging multiple response values ​​within a local neighborhood by taking the maximum value to form a representative value for that neighborhood, or by merging multiple response values ​​within a local neighborhood by taking the average value to form a representative value for that neighborhood. The size of the local neighborhood is set independently. After downsampling by the pooling layer, the spatial scale of the intermediate feature map is reduced while the depth remains unchanged. The output of the pooling layer serves as the input to the next convolutional layer. After processing through all convolutional layers and all pooling layers, the convolutional neural network outputs a feature map. The feature map is a set of two-dimensional matrices arranged along the depth direction. The number of two-dimensional matrices is equal to the number of convolutional kernels in the last convolutional layer. Each two-dimensional matrix corresponds to a channel, and each channel encodes visual features at different levels of abstraction in the bridge surface image.

[0022] Candidate regions are extracted from the feature map, generating multiple candidate regions. The extraction method involves generating multiple reference rectangles centered at each spatial location on the feature map, based on multiple preset reference rectangle sizes and aspect ratios. The reference rectangle size determines the area of ​​the reference rectangle, and the aspect ratio determines its shape. Several discrete area values ​​are selected from the range of 64×64 pixels to 256×256 pixels; for example, three area values ​​are selected: 64×64 pixels, 128×128 pixels, and 256×256 pixels. The aspect ratios are selected from five values: 1:1, 1:2, 2:1, 1:3, and 3:1. The number of reference rectangles generated at each spatial location is equal to the product of the number of selected reference rectangle sizes and the number of selected reference rectangle aspect ratios. All reference bounding boxes generated at all spatial locations constitute a candidate region set. Each candidate region in the candidate region set contains the coordinates of its center point on the feature map, its width, its height, and the size and aspect ratio of the reference bounding box corresponding to that candidate region.

[0023] For each candidate region, bounding box regression and disease category confidence prediction are performed. Specifically, the local features corresponding to each candidate region in the feature map are converted into a fixed-size local feature vector using a region-of-interest (ROI) pooling operation. The RIO pooling operation evenly divides the area covered by the candidate region on the feature map into a fixed number of sub-region grids, for example, 7×7 sub-region grids of equal area. The maximum value of all response values ​​within each sub-region grid is taken, and the maximum values ​​of all sub-region grids are arranged in row-major order to form the local feature vector. The local feature vector is input into the fully connected layer. The fully connected layer performs matrix multiplication between the local feature vector and the weight matrix, adds a bias vector, and then outputs a bounding box offset and a disease category confidence vector via an activation function. The activation function is a modified linear unit function. The bounding box offset includes the lateral offset of the candidate region's center point coordinates, the vertical offset of the candidate region's center point coordinates, a scaling factor for the candidate region's width, and a scaling factor for the candidate region's height. The disease category confidence vector contains the confidence value of the candidate region belonging to each preset disease category. The preset disease categories include six categories: cracks, spalling, honeycomb pitting, exposed reinforcement, efflorescence, and rust. The bounding box offset is used to refine the original position and size of the candidate region. The refined candidate region position is obtained by adding the corresponding offset to the original center point coordinates, and the refined candidate region size is obtained by multiplying the original width and original height by the corresponding scaling factor. The result of disease category confidence prediction is a disease category confidence vector predicted for each candidate region. The preset disease category corresponding to the maximum value in the disease category confidence vector is taken as the disease category predicted for the candidate region, and the maximum value is taken as the disease category confidence of the candidate region.

[0024] Candidate regions with a disease category confidence score higher than a preset confidence threshold are selected as disease rectangle candidate boxes from multiple candidate regions. The preset confidence threshold is set by using a bridge surface image sample set containing ground truth values ​​for labeled disease rectangle candidate boxes. The distribution of disease category confidence scores across all correctly detected disease regions in the bridge surface image sample set by the convolutional neural network is statistically analyzed. The lowest confidence score value that achieves a preset recall rate on the bridge surface image sample set is taken as the preset confidence threshold. The disease category confidence score of each candidate region is compared one by one with the preset confidence threshold. Candidate regions with a disease category confidence score higher than the preset confidence threshold are retained, while candidate regions with a disease category confidence score less than or equal to the preset confidence threshold are discarded. For each candidate region retained after comparison, the refined position and size of the candidate region are used as the bounding box coordinates of the candidate region. The bounding box coordinates are represented by the x-coordinate of the upper left corner of the bounding box, the y-coordinate of the upper left corner of the bounding box, the width of the bounding box, and the height of the bounding box. The bounding box coordinates are calculated based on the coordinates of the center point of the refined candidate region and the size of the refined candidate region.

[0025] Output the candidate bounding boxes for disease lesions and the corresponding disease category confidence score for each candidate bounding box. Each candidate bounding box is a rectangular area defined in the bridge surface image by four coordinates: the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the bounding box, and the height of the bounding box. The disease category confidence score is the maximum value in the disease category confidence vector obtained during the disease category confidence prediction process for each candidate bounding box. The preset disease category corresponding to the disease category confidence score is also retained as the disease category label for the candidate bounding box. The output candidate bounding boxes and their corresponding disease category confidence scores serve as the basis for subsequent processing steps. Each candidate bounding box independently contains three pieces of information: bounding box coordinates, disease category confidence score, and disease category label.

[0026] In the specific implementation of S2, within the candidate bounding box of the defect rectangle, the gray-level co-occurrence matrix is ​​calculated pixel by pixel from the four boundaries of the candidate bounding box into the interior of the candidate bounding box, and the texture contrast of each pixel line is extracted. The candidate bounding box of the defect rectangle is a rectangular area determined in the bridge surface image by the x-coordinate of the upper left corner of the bounding box, the y-coordinate of the upper left corner of the bounding box, the width of the bounding box, and the height of the bounding box output by S1. Within the rectangular area defined by the candidate bounding box of the defect rectangle, using pixels as the basic unit, the upper boundary, lower boundary, left boundary, and right boundary of the candidate bounding box of the defect rectangle are used as starting positions. Starting from each pixel point on each boundary, the process moves pixel by pixel along the direction perpendicular to the boundary of the candidate bounding box and pointing into the interior of the candidate bounding box, generating a set of pixel lines parallel to the boundary and spaced 1 pixel apart. Each pixel line consists of a row or column of pixels arranged sequentially from the boundary pixel along the advancing direction. The length of the pixel line gradually decreases from the boundary inwards, decreasing by 2 pixels for every 1 pixel advanced inwards, ensuring that both ends of the pixel line remain within the candidate bounding box of the lesion rectangle and do not extend beyond the other boundary. For each pixel line, a local neighborhood window centered on each pixel on that pixel line is taken, with a size of 5×5 pixels. The gray-level co-occurrence matrix is ​​calculated within the local neighborhood window. The gray-level co-occurrence matrix is ​​calculated as follows: within a local neighborhood window, the frequency of occurrence of pixel pairs is counted with a preset pixel spacing and a preset direction. The preset pixel spacing is 1 pixel, and the preset directions are 0 degrees, 45 degrees, 90 degrees, and 135 degrees. For each preset direction, the number of times pixel pairs that satisfy the preset pixel spacing and preset direction constraints occur within the local neighborhood window is counted, and the occurrence count is filled into the corresponding position in the gray-level co-occurrence matrix. The row and column numbers of the gray-level co-occurrence matrix correspond to the gray level of the first pixel and the gray level of the second pixel of the pixel pair, respectively. The gray-level co-occurrence matrix is ​​a square matrix, and the dimension of the square matrix is ​​equal to the number of gray levels of the bridge surface image, which is 256. Before calculating the gray-level co-occurrence matrix (GLCM), the gray values ​​of pixels within the local neighborhood window are first compressed, uniformly mapping the gray-level range of 0 to 255 to the gray-level range of 0 to 31. The number of gray levels after compression is set to 32, resulting in a 32×32 square matrix for the GLCM. The GLCM for each preset direction is then normalized by dividing the value of each element by the sum of all elements in the GLCM, ensuring that the sum of all elements in the GLCM equals 1.For the normalized gray-level co-occurrence matrices at 0 degrees, 45 degrees, 90 degrees, and 135 degrees, the texture contrast value for each preset direction is extracted. The extraction method involves iterating through all elements of the gray-level co-occurrence matrix, multiplying the square of the difference between the row and column numbers of each element by the element value, and summing the products to obtain the texture contrast value. The average of the texture contrast values ​​for the four preset directions is then taken as the texture contrast value of the gray-level co-occurrence matrix at the center pixel of that pixel line. This calculation is performed pixel-by-pixel for each center pixel, and the average of the texture contrast values ​​of the gray-level co-occurrence matrix at all center pixels along each pixel line is taken as the texture contrast value for that pixel line. This pixel-by-pixel calculation of the gray-level co-occurrence matrix and extraction of texture contrast is performed on each pixel line along the upper, lower, left, and right boundaries of the defect rectangle candidate box to obtain the texture contrast of each pixel line.

[0027] Along each boundary inwards, the texture contrast of each pixel line is arranged sequentially to obtain the texture contrast change sequence corresponding to each boundary. For the upper boundary, the texture contrast of each pixel line starting from the upper boundary is arranged sequentially in order of increasing depth as it moves inwards from the upper boundary towards the candidate bounding box of the lesion rectangle, forming the upper boundary texture contrast change sequence. For the lower boundary, the texture contrast of each pixel line starting from the lower boundary is arranged sequentially in order of increasing depth as it moves inwards from the lower boundary towards the candidate bounding box of the lesion rectangle, forming the lower boundary texture contrast change sequence. For the left boundary, the texture contrast of each pixel line starting from the left boundary is arranged sequentially in order of increasing depth as it moves inwards from the left boundary towards the candidate bounding box of the lesion rectangle, forming the left boundary texture contrast change sequence. For the right boundary, the texture contrast of each pixel line starting from the right boundary is arranged sequentially in order of increasing depth as it moves inwards from the right boundary towards the candidate bounding box of the lesion rectangle, forming the right boundary texture contrast change sequence. Each position in the texture contrast change sequence corresponds to a pixel line, and the value at each position is the texture contrast of that pixel line. The index number of the texture contrast change sequence starts from 1 and increases. Index number 1 corresponds to the pixel line closest to the boundary, and the larger the index number, the farther the corresponding pixel line is from the boundary.

[0028] The locations of abrupt changes in texture contrast are determined from the texture contrast variation sequence, and the pixel lines corresponding to these locations are used as boundary lines. Specifically, the location of these abrupt changes is determined by subtracting the texture contrast value of the preceding pixel line from the texture contrast value of each pixel line in the texture contrast variation sequence, resulting in a texture contrast difference value. For the pixel line with index u in the texture contrast variation sequence, the texture contrast value of the pixel line with index u is subtracted from the texture contrast value of the pixel line with index u-1, resulting in a texture contrast difference value corresponding to the pixel line with index u. The pixel line with index 1 is not included in the texture contrast difference value calculation. The texture contrast difference value is then compared with a preset abrupt change threshold. The preset abrupt change threshold is set by using a bridge surface image sample set containing the true values ​​of the boundary lines of the labeled core disease areas and disease transition zones. The distribution of texture contrast difference values ​​at all boundary line locations in the bridge surface image sample set is statistically analyzed, and the texture contrast difference value corresponding to the boundary line positioning accuracy on the bridge surface image sample set that meets the preset accuracy requirement is taken as the preset abrupt change threshold. The pixel line corresponding to the first point where the texture contrast difference value exceeds a preset abrupt change threshold is defined as the location of the texture contrast abrupt change. "First exceeding" refers to sequentially traversing the texture contrast difference values ​​in ascending order of their index numbers along the texture contrast change sequence. The pixel line corresponding to the first texture contrast difference value exceeding the preset abrupt change threshold is determined as the location of the texture contrast abrupt change. The pixel line corresponding to the location of the texture contrast abrupt change in the upper boundary texture contrast change sequence is defined as the upper boundary boundary line; the pixel line corresponding to the location of the texture contrast abrupt change in the lower boundary texture contrast change sequence is defined as the lower boundary boundary line; the pixel line corresponding to the location of the texture contrast abrupt change in the left boundary texture contrast change sequence is defined as the left boundary boundary line; and the pixel line corresponding to the location of the texture contrast abrupt change in the right boundary texture contrast change sequence is defined as the right boundary boundary line.

[0029] The area enclosed by the upper, lower, left, and right boundary lines is designated as the core disease area, and the area between the boundary lines and the boundary of the disease candidate rectangle is designated as the disease transition zone. The upper and lower boundary lines are horizontal lines, the left and right boundary lines are vertical lines, and the intersections of each of these boundary lines form a closed rectangle, which is the core disease area. The location of the core defect area in the bridge surface image is determined by the x-coordinate of its upper left corner, the y-coordinate of its upper left corner, its width, and its height. The x-coordinate of the upper left corner equals the x-coordinate of the left boundary line, the y-coordinate equals the y-coordinate of the upper left boundary line, the width equals the x-coordinate of the right boundary line minus the x-coordinate of the left boundary line, and the height equals the y-coordinate of the lower boundary line minus the y-coordinate of the upper boundary line. The area between the upper boundary line and the upper boundary of the defect rectangle candidate box is the upper transition zone; the area between the lower boundary line and the lower boundary of the defect rectangle candidate box is the lower transition zone; the area between the left boundary line and the left boundary of the defect rectangle candidate box is the left transition zone; and the area between the right boundary line and the right boundary of the defect rectangle candidate box is the right transition zone. The upper, lower, left, and right transition zones together constitute the defect transition zone. The division results of the core disease area and the disease transition zone area will serve as the basis for the spatial range of the first texture feature vector and the second texture feature vector in subsequent steps.

[0030] In the specific implementation of S3, a gray-level co-occurrence matrix (GLCM) is calculated for the core defect area. Texture contrast, texture energy, texture entropy, and texture correlation are extracted from the GLCM to form the first texture feature vector. The core defect area is defined by the closed rectangular region enclosed by the upper, lower, left, and right boundary lines in S2. The pixel range of the core defect area in the bridge surface image is defined by the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the width of the core defect area, and the height of the core defect area. The gray values ​​of all pixels within the core defect area are used as input data for calculating the GLCM. Before calculating the GLCM, the gray values ​​of the pixels within the core defect area are compressed, uniformly mapping the gray level range of 0 to 255 to the gray level range of 0 to 31. The number of gray levels after compression is 32. The gray-level co-occurrence matrix (GLCM) is calculated as follows: Within the core disease area, the frequency of pixel pairs is counted based on a preset pixel spacing and preset direction. The preset pixel spacing is 1 pixel, and the preset directions are 0 degrees, 45 degrees, 90 degrees, and 135 degrees (4 directions in total). For each preset direction, the number of times pixel pairs satisfying the preset pixel spacing and direction constraints appear within the core disease area is counted, and this count is entered into the corresponding position in the GLCM. The row number of the GLCM corresponds to the compressed gray level of the first pixel in the pixel pair, and the column number corresponds to the compressed gray level of the second pixel in the pixel pair. The GLCM has a dimension of 32×32. The GLCM for each preset direction is then normalized by dividing the value of each element by the sum of all elements in the GLCM, ensuring that the sum of all elements in the GLCM equals 1. For the normalized gray-level co-occurrence matrices in the 0-degree, 45-degree, 90-degree, and 135-degree directions, texture contrast, texture energy, texture entropy, and texture correlation values ​​are extracted respectively. The texture contrast of the core disease area is obtained by averaging the extracted texture contrast values ​​from the four preset directions; the texture energy of the core disease area is obtained by averaging the extracted texture energy values ​​from the four preset directions; the texture entropy of the core disease area is obtained by averaging the extracted texture entropy values ​​from the four preset directions; and the texture correlation of the core disease area is obtained by averaging the extracted texture correlation values ​​from the four preset directions. The texture contrast value is extracted by iterating through all elements of the gray-level co-occurrence matrix, calculating the square of the difference between the row number and column number of each element, multiplying it by the element value, and summing all the products. The texture energy value is extracted by iterating through all elements of the gray-level co-occurrence matrix, calculating the square of the element value of each element, and summing all the squares.The texture entropy value is extracted by iterating through all elements of the gray-level co-occurrence matrix. For each element with a value greater than zero, the value is multiplied by the logarithm of the element's base 2. The sum of these products is then taken as the negative of the result. The texture correlation value is extracted by calculating the mean of the gray-level co-occurrence matrix along both the row and column directions, as well as the standard deviation of each direction. Then, the matrix iterates through all elements, and for each element, the difference between the row number and the mean of the row direction is multiplied by the difference between the column number and the mean of the column direction, multiplied by the element value. The sum of these products is then divided by the product of the standard deviations of the row and column directions and the sum of the element values. The texture contrast, texture energy, texture entropy, and texture correlation of the core disease area are arranged in a fixed order, with texture contrast in the first position, texture energy in the second position, texture entropy in the third position, and texture correlation in the fourth position, to form a four-dimensional vector, which is the first texture feature vector.

[0031] A gray-level co-occurrence matrix (GLCM) is calculated for the disease transition zone region. Texture contrast, texture energy, texture entropy, and texture correlation are extracted from the GLCM to form a second texture feature vector. The disease transition zone region is composed of the upper, lower, left, and right transition zones in S2. The upper transition zone is the pixel region between the upper boundary line and the upper boundary of the disease rectangle candidate box; the lower transition zone is the pixel region between the lower boundary line and the lower boundary of the disease rectangle candidate box; the left transition zone is the pixel region between the left boundary line and the left boundary of the disease rectangle candidate box; and the right transition zone is the pixel region between the right boundary line and the right boundary of the disease rectangle candidate box. The gray values ​​of all pixels within the disease transition zone region are used as input data for calculating the GLCM. All pixels within the disease transition zone region refer to the union of all pixels contained in the upper, lower, left, and right transition zones. Before calculating the gray-level co-occurrence matrix, the gray-level values ​​of pixels in the disease transition zone are compressed using the same gray-level compression operation as those in the core disease zone, uniformly mapping the gray-level range of 0 to 255 to the gray-level range of 0 to 31. The number of compressed gray-levels is set to 32. Within the disease transition zone, the frequency of occurrence of pixel pairs is counted according to preset pixel spacing and preset directions. The preset pixel spacing is 1 pixel, and the preset directions are 0 degrees, 45 degrees, 90 degrees, and 135 degrees. For each preset direction, the number of times pixel pairs satisfying the preset pixel spacing and preset direction constraints occur in the disease transition zone is counted. The occurrence count is then entered into the corresponding position in the gray-level co-occurrence matrix. The row number of the gray-level co-occurrence matrix corresponds to the compressed gray-level of the first pixel in the pixel pair, and the column number corresponds to the compressed gray-level of the second pixel in the pixel pair. The dimension of the gray-level co-occurrence matrix is ​​32×32. For each preset direction's gray-level co-occurrence matrix, the same normalization process as for the core disease region is performed, ensuring the sum of all elements in the gray-level co-occurrence matrix equals 1. For the normalized gray-level co-occurrence matrices in the 0-degree, 45-degree, 90-degree, and 135-degree directions, texture contrast, texture energy, texture entropy, and texture correlation values ​​are extracted, using the same extraction method as for the core disease region. The extraction methods for texture contrast, texture energy, texture entropy, and texture correlation values ​​are also consistent with those used to calculate the corresponding values ​​for the core disease region. The texture contrast of the disease transition zone is obtained by averaging the extracted texture contrast, texture energy, texture entropy, and texture correlation values ​​from the four preset directions.The texture contrast, texture energy, texture entropy, and texture correlation of the disease transition zone are combined into a four-dimensional vector in a fixed order: texture contrast first, texture energy second, texture entropy third, and texture correlation fourth. This four-dimensional vector is the second texture feature vector. The first and second texture feature vectors serve as the basis for subsequent steps of calculating texture realism and vector difference.

[0032] In the specific implementation of S4, the texture contrast values ​​of the core disease area and the disease transition zone area are extracted point by point along the boundary lines. The boundary lines include four lines determined in S2: the upper boundary line, the lower boundary line, the left boundary line, and the right boundary line. For each boundary line, the sampling points are selected by starting from one endpoint of the boundary line and sampling point by point along the extension direction of the boundary line with a step size of 1 pixel. Each sampling point corresponds to a pixel position on the boundary line. For the upper boundary line, the sampling points start from the left endpoint of the upper boundary line and are sampled point by point along the direction from left to right with a step size of 1 pixel, resulting in the upper boundary line sampling point sequence. For the lower boundary line, the sampling points start from the left endpoint of the lower boundary line and are sampled point by point along the direction from left to right with a step size of 1 pixel, resulting in the lower boundary line sampling point sequence. For the left boundary line, sampling points are taken starting from the upper endpoint of the left boundary line and proceeding downwards in a step of 1 pixel, resulting in a sampling point sequence for the left boundary line. For the right boundary line, sampling points are taken starting from the upper endpoint of the right boundary line and proceeding downwards in a step of 1 pixel, resulting in a sampling point sequence for the right boundary line. At each sampling point, the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area are extracted. The texture contrast values ​​of the core disease area and the disease transition zone area are both derived from the component values ​​of the texture feature vector calculated in S3, where the texture contrast value of the core disease area is the first element value in the first texture feature vector, and the texture contrast value of the disease transition zone area is the first element value in the second texture feature vector. The above extraction operation is performed on each sampling point on each boundary line to obtain the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area corresponding to each sampling point on each boundary line.

[0033] Calculate the absolute value of the difference between the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area. Arrange the absolute values ​​according to the sampling points of the boundary line to obtain the difference sequence. For each boundary line, the absolute value of the difference between the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area at each sampling point on the boundary line is taken as the texture contrast difference value of that sampling point. For the upper boundary line, arrange the texture contrast difference values ​​of each sampling point in the upper boundary line sampling point sequence in order from the left endpoint to the right endpoint of the upper boundary line to obtain the upper boundary difference sequence. For the lower boundary line, arrange the texture contrast difference values ​​of each sampling point in the lower boundary line sampling point sequence in order from the left endpoint to the right endpoint of the lower boundary line to obtain the lower boundary difference sequence. For the left boundary line, arrange the texture contrast difference values ​​of each sampling point in the left boundary line sampling point sequence in order from the upper endpoint to the lower endpoint of the left boundary line to obtain the left boundary difference sequence. For the right boundary line, the texture contrast difference values ​​of each sampling point in the right boundary line sampling point sequence are arranged sequentially from the upper endpoint to the lower endpoint of the right boundary line to obtain the right boundary difference sequence. The upper boundary difference sequence, right boundary difference sequence, lower boundary difference sequence, and left boundary difference sequence are concatenated end to end in clockwise order to obtain a closed difference sequence. The length of the difference sequence is equal to the sum of the number of sampling points of the upper boundary line, the lower boundary line, the left boundary line, and the right boundary line.

[0034] The local entropy of the difference sequence is calculated and compared with a preset entropy threshold. If the local entropy is lower than the preset threshold, the texture fidelity / phasing is determined to be high; otherwise, it is determined to be low. The local entropy of the difference sequence is calculated by setting a sliding window on the difference sequence. The length of the sliding window is a preset proportion of the difference sequence length. The sliding window slides across the difference sequence point by point with a step size of one sampling point, and the length of the sliding window is measured in units of sampling points. The preset proportion is set by testing the impact of different sliding window lengths on the accuracy of texture fidelity / phasing on a bridge surface image sample set. The ratio of the sliding window length that maximizes the accuracy of texture fidelity / phasing to the difference sequence length is taken as the preset proportion; for example, a preset proportion of 0.2 is used. For each position covered by the sliding window on the difference sequence, the information entropy of all texture contrast difference values ​​within the sliding window is calculated. The information entropy is calculated by obtaining the minimum and maximum values ​​of the texture contrast difference values ​​within the sliding window, and then uniformly dividing the range from the minimum to the maximum value into a predetermined number of intervals. The predetermined number of intervals is determined based on the total length of the difference sequence, taking the integer part of the square root of the total length. Each texture contrast difference value within the sliding window is assigned to its corresponding interval according to its value. The number of texture contrast difference values ​​falling into each interval is counted, and the proportion of the number of texture contrast difference values ​​falling into each interval to the total number of texture contrast difference values ​​within the sliding window is calculated. For each interval with a proportion greater than zero, the proportion is multiplied by the logarithm of the proportion to the base 2. The sum of the products of all intervals satisfying the condition is then taken as the negative, yielding the information entropy of all texture contrast difference values ​​within the sliding window. The average of all information entropy values ​​obtained by sliding the sliding window point by point on the difference sequence is taken; this average value is the local entropy of the difference sequence. The preset entropy threshold is set as follows: using a bridge surface image sample set containing labeled texture fidelity values, the upper quantile of the local entropy of all difference sequences with high texture fidelity values ​​and the lower quantile of the local entropy of all difference sequences with low texture fidelity values ​​are calculated. The midpoint between the upper and lower quantiles is taken as the preset entropy threshold. The calculated local entropy of the difference sequences is compared with the preset entropy threshold. If the local entropy of the difference sequence is less than the preset entropy threshold, the texture fidelity is determined to be high, for example, 1.0; if the local entropy of the difference sequence is not less than the preset entropy threshold, the texture fidelity is determined to be low, for example, 0.15.

[0035] A weighted second texture feature vector is obtained by performing a scalar multiplication operation on the texture fidelity value. The scalar multiplication operation is performed by using the texture fidelity value as a scalar factor, multiplying each element of the second texture feature vector by the scalar factor, and then using the results to form the weighted second texture feature vector in the element-wise order of the second texture feature vector. The second texture feature vector is a four-dimensional vector extracted from S3, composed of the texture contrast, texture energy, texture entropy, and texture correlation of the disease transition zone. The weighted second texture feature vector has the same dimension as the second texture feature vector, and the order of the elements in the weighted second texture feature vector is consistent with the order of the corresponding elements in the second texture feature vector. When the texture fidelity value is high, the value of each element in the weighted second texture feature vector is equal to the value of the corresponding element in the second texture feature vector multiplied by the high value of the texture fidelity value. When the texture fidelity is low, the value of each element in the weighted second texture feature vector is equal to the value of the corresponding element in the second texture feature vector multiplied by the low value of the texture fidelity. The weighted second texture feature vector will be used as input data for subsequent steps to calculate the vector difference.

[0036] In the specific implementation of S5, the cosine distance between the first texture feature vector and the weighted second texture feature vector is calculated, and the cosine distance is used as the vector difference degree. The first texture feature vector is the four-dimensional vector extracted in S3, composed of the texture contrast, texture energy, texture entropy, and texture correlation of the core disease region. The weighted second texture feature vector is the vector output in S4, obtained by performing a scalar multiplication operation on the second texture feature vector. The dimension of the weighted second texture feature vector is the same as that of the first texture feature vector. The cosine distance is calculated as: d = 1 − (A·B) / (‖A‖×‖B‖); where d represents the cosine distance, A represents the first texture feature vector, B represents the weighted second texture feature vector, A·B represents the vector dot product of the first texture feature vector and the weighted second texture feature vector, ‖A‖ represents the magnitude of the first texture feature vector, and ‖B‖ represents the magnitude of the weighted second texture feature vector. The vector dot product is calculated as: A·B = Σ(Ai × Bi); where Ai represents the i-th element in the first texture feature vector, Bi represents the i-th element in the weighted second texture feature vector, and Σ represents the summation over i from 1 to the total number of vector dimensions. The modulus of the first texture feature vector is calculated as follows: Among them, Ai 2 Let represent the square of the i-th element in the first texture feature vector. The modulus of the weighted second texture feature vector is calculated as follows: Among them, Bi 2This represents the square of the i-th element in the weighted second texture feature vector. The calculated cosine distance is used as the vector difference, which ranges from zero to two. The closer the vector difference is to zero, the closer the first texture feature vector and the weighted second texture feature vector are in the vector space. The closer the vector difference is to two, the more opposite the directions of the first texture feature vector and the weighted second texture feature vector are in the vector space.

[0037] The vector difference is compared with a difference threshold. The difference threshold is set by using a bridge surface image sample set containing ground truth values ​​for whether or not transition zone texture contamination occurs. The vector difference distributions of all transition zone texture contamination samples and all samples without transition zone texture contamination are statistically analyzed. The vector difference value that achieves the preset accuracy requirement for distinguishing between samples with and without transition zone texture contamination is taken as the difference threshold. Specifically, the calculated vector difference is compared with the difference threshold. If the vector difference is greater than the difference threshold, it is determined that the weighted second texture feature vector has transition zone texture contamination over the first texture feature vector, and a weighting reduction coefficient is calculated based on the vector difference. If the vector difference is not greater than the difference threshold, it is determined that the influence of the weighted second texture feature vector on the first texture feature vector is within an acceptable range, and the first texture feature vector is used as the corrected texture feature vector.

[0038] If the vector difference exceeds the difference threshold, a weighting coefficient is calculated based on the vector difference. This weighting coefficient is then multiplied by the first texture feature vector to obtain the corrected texture feature vector. The trigger condition for calculating the weighting coefficient is that the vector difference is greater than the difference threshold, and the calculation of the weighting coefficient depends on the magnitude of the vector difference. The vector difference is input into the weighting function, which is a non-linear function that monotonically decreases the vector difference from the difference threshold to the upper limit of the difference. The weighting coefficient is calculated as: w = f(Dd); where w represents the weighting coefficient, Dd represents the vector difference, f represents the weighting function, the input of the weighting function is the vector difference, and the output is the weighting coefficient. The upper limit of the difference is set by statistically analyzing the maximum value of the vector difference of all transition zone texture contamination samples in the bridge surface image sample set, rounding this maximum value up to a preset scale interval to obtain the upper limit of the difference. For example, the scale interval is 0.5. The weighting function outputs a weighting coefficient close to 1.0 at the difference threshold and close to the minimum weighting coefficient at the difference upper limit. The minimum weighting coefficient is determined by observing the impact of different minimum weighting coefficient values ​​on the accuracy of texture feature vectors used for texture classification on a bridge surface image sample set, and then selecting the minimum weighting coefficient value that maximizes the accuracy; for example, the minimum weighting coefficient is set to 0.1. The monotonically decreasing nonlinear function is constructed with vector difference as the independent variable and the weighting coefficient as the dependent variable. The function curve smoothly decreases within the difference threshold to the difference upper limit, and the rate of decrease gradually slows down as the vector difference increases. After inputting the vector difference into the monotonically decreasing nonlinear function, the weighting function outputs the weighting coefficient, which ranges from zero to one and is a floating-point number. The weighting coefficients are multiplied element-wise with the first texture feature vector. Each element in the first texture feature vector is multiplied by the weighting coefficient. The result of the multiplication is used to form a corrected texture feature vector, which has the same dimension as the first texture feature vector, and the order of the elements in the corrected texture feature vector is consistent with the order of the corresponding elements in the first texture feature vector. The corrected texture feature vector will be used as the feature data input to the texture classifier in subsequent steps.

[0039] In the specific implementation of S6, the modified texture feature vector is input into the texture classifier. The texture classifier calculates the similarity between the modified texture feature vector and the pre-stored disease texture category templates, and outputs the disease texture category corresponding to the pre-stored disease texture category template with the highest similarity. The texture classifier is a pre-trained classifier used for disease texture category discrimination. The training method of the texture classifier is as follows: collect bridge surface disease texture image samples containing six disease texture categories, including cracks, peeling, honeycomb pitting, exposed reinforcement, efflorescence, and rust. For each bridge surface disease texture image sample, the modified texture feature vector is extracted according to steps S1 to S5. The modified texture feature vector and the corresponding disease texture category label are used as training data pairs. A classification model is trained using supervised learning. The classification model adopts the support vector machine model. The kernel function of the support vector machine is the radial basis function. During the training process, the width parameter and penalty coefficient of the radial basis function are selected by cross-validation to maximize the texture classification accuracy of the trained support vector machine model on the validation set. After training, the texture classifier is obtained. The pre-stored disease texture category templates are established as follows: For each disease texture category, all modified texture feature vectors belonging to that disease texture category are selected from the training data pairs. The arithmetic mean of all modified texture feature vectors in each dimension is calculated to obtain an average feature vector with the same dimensions as the modified texture feature vectors. This average feature vector is used as the pre-stored disease texture category template for that disease texture category. The number of pre-stored disease texture category templates is equal to the number of disease texture categories. The texture classifier calculates the similarity between the modified texture feature vectors and the pre-stored disease texture category templates by calculating the cosine similarity between the modified texture feature vectors and each pre-stored disease texture category template. All calculated cosine similarity values ​​are sorted in descending order, and the disease texture category corresponding to the pre-stored disease texture category template with the largest cosine similarity value is taken as the disease texture category output by the texture classifier. The disease texture category is one of the following: crack, peeling, honeycomb pitting, exposed reinforcement, efflorescence, or rust.

[0040] The comparison checks whether the disease texture category matches the disease category corresponding to the disease category confidence score. The disease category corresponding to the disease category confidence score is the disease category recorded in the disease category label corresponding to each disease rectangle candidate box output by S1. The disease category recorded in the disease category label is one of the preset disease categories predicted and output by the convolutional neural network in S1. The preset disease categories include six categories: cracks, peeling, honeycomb pitting, exposed veins, efflorescence, and rust. The specific comparison operation is as follows: the disease texture category output by the texture classifier is compared with the disease category corresponding to the disease category confidence score. If the name strings of the disease texture category and the disease category corresponding to the disease category confidence score are exactly the same, then the disease texture category and the disease category corresponding to the disease category confidence score are considered to be consistent. If the name strings of the disease texture category and the disease category corresponding to the disease category confidence score are different, then the disease texture category and the disease category corresponding to the disease category confidence score are considered to be inconsistent.

[0041] If the disease texture category matches the disease category corresponding to the disease category confidence score, then the disease category confidence score is increased as the final confidence score. The method for increasing the disease category confidence score is to multiply it by a preset increase coefficient, which is a floating-point number greater than 1.0. The increase coefficient is set by using a bridge surface image sample set containing labeled disease texture category ground truth values. When all disease texture categories in the bridge surface image sample set match the disease category corresponding to the disease category confidence score predicted by the convolutional neural network, the average increase ratio between the predicted disease category confidence score and the true confidence score is calculated. This average increase ratio is taken as the increase coefficient, for example, 1.25. The value obtained by multiplying the disease category confidence score by the increase coefficient is used as the final confidence score. If the multiplied value exceeds 1.0, the final confidence score is truncated to 1.0; if the multiplied value does not exceed 1.0, the multiplied value remains unchanged. If the disease texture category and the disease category corresponding to the disease category confidence score are inconsistent, the disease texture category will replace the disease category corresponding to the disease category confidence score, and the replaced disease category confidence score will be used as the final confidence score. Specifically, the replacement operation involves updating the disease category field in the disease category label corresponding to the disease rectangle candidate box output by S1 to the disease texture category output by the texture classifier. The value of the disease category confidence score field corresponding to the disease rectangle candidate box remains unchanged. The updated disease category confidence score corresponding to the disease rectangle candidate box is the final confidence score.

[0042] The final classification result is output, which includes the disease texture category and the final confidence score. The output format of the final classification result is a pair of data. The first item of the pair is the disease texture category, and the second item is the final confidence score. The final classification result also includes the location information of the disease rectangle candidate box in the bridge surface image. The location information consists of the x-coordinate of the top-left corner of the disease rectangle candidate box output by S1, the y-coordinate of the top-left corner of the bounding box, the width of the bounding box, and the height of the bounding box. The final classification result is used to show bridge inspectors the disease texture category to which the bridge surface image within the range of the disease rectangle candidate box belongs, as well as the reliability of the classification result.

[0043] In this embodiment, a visible light camera, such as that mounted on a bridge defect scanning drone, can be used to photograph the surfaces of components like bridge piers and beam bottoms. The captured digital images are then used as input for the bridge surface images acquired in S1. In the convolutional neural network defect detection step of S1, a YOLOv3 architecture convolutional neural network can be used to detect defects in the bridge surface images. YOLOv3 performs multi-layer convolution and pooling processing on the input images through a feature extraction network, outputting defect candidate rectangles and corresponding defect category confidence scores at multiple scales. The defect category corresponding to the defect category confidence score can cover common bridge concrete surface defects such as cracks, spalling, and honeycomb pitting. After the image enters S2, the texture contrast is calculated within the candidate bounding box of the lesion using the gray-level co-occurrence matrix algorithm, and the boundary line is determined based on the change in texture contrast, separating the core lesion area from the lesion transition zone area. Subsequently, the texture feature vector is extracted in S3, the texture virtuality is determined and weighted in S4 using local entropy, the vector difference is compared and weighted correction is performed in S5, and the texture classifier is used for classification and confidence correction in S6. Finally, the classification result containing the lesion texture category and the final confidence score is output, realizing the extraction and classification of texture features of the lesion image.

[0044] Example 2: Figure 2 A schematic diagram of the structure of a bridge defect image texture feature extraction and classification system according to the present invention is provided. The bridge defect image texture feature extraction and classification system includes: The detection module is used to acquire images of the bridge surface, detect defects using a convolutional neural network, and output candidate rectangular boxes for defects and confidence scores for defect categories. The traceability and segmentation module is used to calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel within the candidate bounding box of the disease rectangle from each boundary inward. Based on the change in texture contrast, the boundary line between the disease and the transition zone is determined. The area inside the boundary line is taken as the core disease area, and the remaining area is taken as the disease transition zone area. The feature extraction module is used to extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; The virtual-real weighting module is used to calculate the difference in texture contrast between the core disease area and the disease transition zone area point by point along the boundary line to obtain the difference sequence, calculate the local entropy of the difference sequence to determine the virtual-real degree of texture, and weight the second texture feature vector with the virtual-real degree of texture to obtain the weighted second texture feature vector; The weight reduction correction module is used to calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is weighted and corrected to obtain the corrected texture feature vector. The classification output module is used to input the corrected texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by correcting the confidence of the disease category.

[0045] All calculations involved in the embodiments are performed using dimensionless numerical values, and the preset parameters and thresholds in the calculations can be set by those skilled in the art according to actual conditions.

[0046] This technical solution can be flexibly deployed, for example, as embedded software running on device hardware, or installed on personal computers or other smart terminals with user interfaces, thus adapting to various hardware environments and usage requirements.

[0047] The above solutions can be implemented in software, hardware, firmware, or a combination thereof. When implemented in software, they are presented, in whole or in part, as a computer program product, containing one or more computer instructions or programs. When these instructions or programs are loaded and executed on a computer, results are produced corresponding to the processes or functions of the embodiments of this application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one medium to another via wired or wireless means, such as from a website, server, or data center via wired means such as fiber optic cables, twisted-pair cables, or coaxial cables, or wireless means such as infrared or microwaves to another site. A computer-readable storage medium refers to any usable medium that a computer can access or a data storage device such as a server or data center that contains one or more usable media, including magnetic media such as floppy disks, hard disks, and magnetic tapes, optical media such as DVDs, and semiconductor media such as solid-state drives.

[0048] The specific working process of the system, device and module can be found in the method embodiment, and will not be repeated here.

[0049] The disclosed systems, devices, and methods can be implemented in other ways. The device embodiments are for illustrative purposes only, and the module division is only a logical division. In practice, different divisions can be implemented, such as merging or integrating multiple modules or components, or omitting some features. Coupling, direct coupling, or communication connections between the components can be achieved through interfaces, while indirect coupling or communication connections can take electrical, mechanical, or other forms.

[0050] The modules described as separate components may or may not be physically separated. The components shown as modules can be physical hardware or software, and can be deployed centrally or distributed across multiple network nodes. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0051] The functional modules in each embodiment can be integrated into one processing module, or they can exist independently, or two or more modules can be integrated into one.

[0052] If the functionality is implemented as a software module and used as an independent product, it can be stored in a computer-readable storage medium. Under this understanding, the substantial contribution of the technical solution of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and contains instructions to cause a computer device, such as a personal computer, server, or network device, to execute all or part of the steps of the methods in the embodiments of this application. The storage medium includes any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0053] The above are merely specific embodiments of this application, and the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should fall within the scope of protection of this application.

Claims

1. A method for extracting and classifying texture features from bridge defect images, characterized in that, include: S1: Acquire bridge surface images, use convolutional neural networks to detect defects, and output candidate rectangular boxes for defects and confidence scores for defect categories; S2: Within the candidate bounding box of the lesion rectangle, calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel from the boundary inwards. Determine the boundary line between the lesion and the transition zone based on the change in texture contrast. The area within the boundary line is designated as the core lesion area, and the remaining area is designated as the lesion transition zone area, including: Within the candidate bounding box of the disease rectangle, the gray-level co-occurrence matrix is ​​calculated pixel by pixel from the four boundaries of the candidate bounding box into the inside of the candidate bounding box, and the texture contrast of each pixel line is extracted. Along each boundary inwards, the texture contrast of each pixel line is arranged sequentially to obtain the texture contrast change sequence corresponding to each boundary; The locations where abrupt changes in texture contrast are determined from the texture contrast change sequence, and the pixel lines corresponding to the locations of abrupt changes are used as boundary lines. The area enclosed by the dividing lines corresponding to the four boundaries is taken as the core disease area, and the area between the dividing lines and the boundary of the disease rectangle candidate box is taken as the disease transition zone area. S3: Extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; S4: Calculate the texture contrast difference between the core disease area and the transition zone area point by point along the boundary line to obtain a difference sequence. Calculate the local entropy of the difference sequence to determine the texture fidelity / reality. Weight the second texture feature vector with the texture fidelity / reality to obtain a weighted second texture feature vector, including: Extract the texture contrast values ​​of the core disease area and the texture contrast value of the disease transition zone area by sampling point along the boundary line; Calculate the absolute value of the difference between the texture contrast value of the core disease area and the texture contrast value of the disease transition zone area, and arrange the absolute values ​​in the order of the sampling points of the boundary line to obtain the difference sequence. Calculate the local entropy of the difference sequence and compare the local entropy with a preset entropy threshold. If the local entropy is lower than the preset entropy threshold, the texture fidelity is determined to be high; if the local entropy is not lower than the preset entropy threshold, the texture fidelity is determined to be low. A scalar multiplication operation is performed on the second texture feature vector using the texture fidelity / reality to obtain a weighted second texture feature vector; S5: Calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is reduced in weight to obtain the corrected texture feature vector. S6: Input the modified texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by modifying the confidence of the disease category.

2. The method for extracting and classifying texture features from bridge defect images according to claim 1, characterized in that, S1 includes: Obtain bridge surface images and input them into a convolutional neural network; Convolutional neural networks perform convolution and pooling processing on bridge surface images to output feature maps; Candidate regions are extracted from the feature map to generate multiple candidate regions; For each candidate region, perform bounding box regression and disease category confidence prediction; From multiple candidate regions, select candidate regions whose disease category confidence is higher than a preset confidence threshold as disease rectangular candidate boxes; Output disease candidate rectangles and the disease category confidence score for each disease candidate rectangle.

3. The method for extracting and classifying texture features from bridge defect images according to claim 1, characterized in that, Determining the location where a sudden change in texture contrast occurs from a texture contrast change sequence includes: subtracting the texture contrast value of the previous pixel line from the texture contrast value of each pixel line in the texture contrast change sequence to obtain a texture contrast difference value; comparing the texture contrast difference value with a preset change threshold; and identifying the pixel line corresponding to the location where the texture contrast difference value first exceeds the preset change threshold as the location where a sudden change in texture contrast occurs.

4. The method for extracting and classifying texture features from bridge defect images according to claim 1, characterized in that, S3 includes: Calculate the gray-level co-occurrence matrix for the core disease area, and extract texture contrast, texture energy, texture entropy and texture correlation from the gray-level co-occurrence matrix to form the first texture feature vector; The gray-level co-occurrence matrix is ​​calculated for the disease transition zone region. Texture contrast, texture energy, texture entropy and texture correlation are extracted from the gray-level co-occurrence matrix to form the second texture feature vector.

5. The method for extracting and classifying texture features from bridge defect images according to claim 1, characterized in that, S5 include: Calculate the cosine distance between the first texture feature vector and the weighted second texture feature vector, and use the cosine distance as the vector difference. Compare the vector dissimilarity to the dissimilarity threshold; If the vector difference exceeds the difference threshold, a weighting coefficient is calculated based on the vector difference, and the weighting coefficient is multiplied by the first texture feature vector to obtain the corrected texture feature vector. If the vector difference does not exceed the difference threshold, then the first texture feature vector is used as the corrected texture feature vector.

6. The method for extracting and classifying texture features from bridge defect images according to claim 5, characterized in that, Calculating the weighting coefficient based on the vector dissimilarity includes: inputting the vector dissimilarity into the weighting function, which is a non-linear function that monotonically decreases the vector dissimilarity from the dissimilarity threshold to the dissimilarity upper limit; and outputting the weighting coefficient from the weighting function, with the value of the weighting coefficient ranging from zero to one.

7. The method for extracting and classifying texture features from bridge defect images according to claim 1, characterized in that, S6 include: The modified texture feature vector is input into the texture classifier. The texture classifier calculates the similarity between the modified texture feature vector and the pre-stored disease texture category template. The disease texture category corresponding to the pre-stored disease texture category template with the highest similarity is taken as the output. Compare whether the disease texture category and the disease category corresponding to the disease category confidence level are consistent; If the disease texture category matches the disease category corresponding to the disease category confidence score, then increase the disease category confidence score as the final confidence score; If the disease texture category is inconsistent with the disease category corresponding to the disease category confidence score, then the disease texture category will replace the disease category corresponding to the disease category confidence score, and the confidence score of the replaced disease category will be used as the final confidence score. Output the final classification result, which includes the disease texture category and the final confidence score.

8. A bridge defect image texture feature extraction and classification system, used to implement the bridge defect image texture feature extraction and classification method according to any one of claims 1-7, characterized in that, include: The detection module is used to acquire images of the bridge surface, detect defects using a convolutional neural network, and output candidate rectangular boxes for defects and confidence scores for defect categories. The traceability and segmentation module is used to calculate the texture contrast of the gray-level co-occurrence matrix pixel by pixel within the candidate bounding box of the disease rectangle from each boundary inward. Based on the change in texture contrast, the boundary line between the disease and the transition zone is determined. The area inside the boundary line is taken as the core disease area, and the remaining area is taken as the disease transition zone area. The feature extraction module is used to extract the first texture feature vector of the core disease area and the second texture feature vector of the disease transition zone area; The virtual-real weighting module is used to calculate the difference in texture contrast between the core disease area and the disease transition zone area point by point along the boundary line to obtain the difference sequence, calculate the local entropy of the difference sequence to determine the virtual-real degree of texture, and weight the second texture feature vector with the virtual-real degree of texture to obtain the weighted second texture feature vector; The weight reduction correction module is used to calculate the vector difference between the first texture feature vector and the weighted second texture feature vector. If the vector difference exceeds the difference threshold, the first texture feature vector is weighted and corrected to obtain the corrected texture feature vector. The classification output module is used to input the corrected texture feature vector into the texture classifier to obtain the disease texture category, and output the final classification result by correcting the confidence of the disease category.

Images