A bridge defect detection method and system based on image recognition

Abstract

The application discloses a kind of bridge defect detection method and system based on image recognition, specifically related to bridge defect detection technical field, the quality of image block dataset is optimized by image acquisition, gradient calculation and multi-scale division processing.Adopting adaptive reconstruction algorithm optimizes pixel arrangement, extracts local feature and analyzes regional feature correlation, dynamically adjusts defect calibration, and finally generates accurate defect position dataset.The method combines multi-scale feature extraction and dynamic adjustment, which can improve the accuracy of defect recognition.The bridge defect detection method and system based on image recognition disclosed in the application significantly improve the accuracy and reliability of bridge defect detection through multi-scale processing and adaptive reconstruction algorithm.The accurate defect positioning and dynamic calibration mechanism reduces the false detection and missed detection in traditional methods, effectively deals with the defect recognition problem under complex background, and provides more accurate and efficient support for bridge maintenance.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a bridge defect detection method and system based on image recognition. Background Technology

[0002] Image recognition technology encompasses a process of acquiring, preprocessing, extracting features, matching patterns, classifying and recognizing objects, defects, structures, or scenes from digital images. The core components of this technology include image acquisition devices and sensors, image enhancement and denoising, edge or texture feature extraction, image region segmentation and target detection, morphological or structural analysis, category or state discrimination, and output of recognition results and decision support. Overall, this technology systematically covers four key stages: image acquisition, image processing, feature recognition, and decision-making and output, and is a crucial means of achieving "machine vision" or "automated image analysis" functions.

[0003] Among them, image recognition-based bridge defect detection methods refer to detection methods that use image recognition technology to identify the surface or internal defect status of bridge structures. This encompasses defect image acquisition, image preprocessing of defect areas, extraction of defect features from the preprocessed image, matching or classifying the extracted defect features with known defect categories, and locating the defect based on the recognition results. Specifically, it involves a process of bridge defect identification using image recognition methods including image acquisition, image processing, feature extraction, and pattern matching.

[0004] In existing technologies, image recognition typically relies on traditional image processing methods, such as edge detection and simple texture analysis, which have limitations in the accuracy of feature extraction and target detection. Image preprocessing and feature extraction are often significantly affected by noise, making it difficult to effectively distinguish between subtle defects and noisy areas, thus reducing recognition accuracy. Furthermore, existing methods lack multi-scale image processing capabilities, leading to potentially large deviations in processing results when faced with defects of different sizes and types, and making them unsuitable for complex and variable bridge structures. During defect location calibration, the lack of dynamic adjustment and real-time verification mechanisms can result in deviations in calibration results, and it is difficult to maintain stable recognition performance under different conditions, making it susceptible to environmental factors or changes in image quality. For example, in complex backgrounds, existing technologies struggle to accurately identify small cracks or corrosion areas, thus affecting subsequent decision-making and maintenance work. Summary of the Invention

[0005] The main objective of this invention is to provide a bridge defect detection method and system based on image recognition. By using multi-scale image processing, adaptive reconstruction algorithms, and regional feature correlation analysis, this invention addresses the shortcomings of existing bridge defect detection methods in terms of accuracy, adaptability, and calibration accuracy, thereby improving the accuracy and reliability of defect identification and location.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A bridge defect detection method based on image recognition, the method comprising: The bridge structure to be inspected is photographed to obtain surface image data of the bridge structure and to obtain an image patch dataset. By analyzing the image patch dataset through gradient calculation, regions with high contrast and / or significant edge changes in the image are identified. Based on the regional characteristics, the images are divided into multiple scales. An adaptive reconstruction algorithm is applied to each image patch to optimize the pixel arrangement, resulting in a multi-scale reconstructed image patch dataset. Local features are extracted from each image patch in the multi-scale reconstructed image patch dataset, and the correlation between local features and surrounding areas is analyzed to generate a region feature correlation matrix; the local features include edge intensity, texture features, and brightness contrast. Based on the regional feature correlation matrix, the initial location of the defect is determined, and the defect calibration is dynamically adjusted through feature correlation analysis to obtain accurate defect location data; Based on accurate defect location data, the calibration results are verified, and the deviations are adjusted to obtain the defect calibration result dataset.

[0007] Preferably, the gradient algorithm is specifically the Laplacian operator; More preferably, the multi-scale segmentation based on regional features specifically involves: for image blocks containing high-contrast regions, dynamically calculating the optimal size of the image block based on the edge strength and feature density of the region, setting a maximum image block size limit, and using smaller image block segmentation; for background regions, setting a minimum image block size limit, and using larger image block segmentation to obtain a multi-scale segmented image dataset.

[0008] Preferably, an adaptive reconstruction algorithm is applied to each image patch to optimize the pixel arrangement, resulting in a multi-scale reconstructed image patch dataset, specifically: After obtaining the multi-scale segmented image dataset, local features are calculated for each image patch to identify background and / or defect regions. For defect regions, local mean filtering is used to adjust the weight of each pixel to enhance defect features. For background regions, smoothing is used to reduce noise interference. Pixels are rearranged and reconstructed image patches are output to obtain the multi-scale reconstructed image patch dataset.

[0009] Preferably, the step of extracting local features of each image patch in the multi-scale reconstructed image patch dataset, analyzing the correlation between local features and surrounding areas, and generating a region feature correlation matrix includes: The edge intensity of each image patch is calculated, its texture features and brightness contrast information are extracted, and then zero-mean unit variance standardization is performed to obtain the local feature dataset. Based on the local feature dataset, Gaussian filters are used to perform spatial filtering on each image patch, Canny edge detection is used to identify significant edges in the image, and K-means clustering is used to analyze the spatial correlation between local features in the image patch and the surrounding area to obtain the optimized feature dataset. Based on the optimized feature dataset, the local features of each image patch are matched with a preset standard template, their feature similarity is calculated, and the features most relevant to the defect area are selected. Finally, the defect area is identified, and a regional feature association matrix is ​​generated.

[0010] Preferably, the standard template includes edge features, texture features, brightness contrast features, color features, and defect-specific template features, including crack template features, corrosion template features, and peeling template features.

[0011] Preferably, the step of determining the preliminary defect location based on the regional feature correlation matrix and dynamically adjusting the defect calibration through feature correlation analysis to obtain precise defect location data includes: Based on the regional feature correlation matrix, the edge features and texture features of the defect area are extracted. Through image segmentation and contour matching methods, the location and orientation of the defect are initially determined, and preliminary defect location data is generated. Based on the preliminary defect location data, similarity analysis and optical flow method are used to calculate the defect location change trend. Combined with the spatial relationship of adjacent image blocks, the defect location calibration is dynamically adjusted to obtain defect location change trend data. Based on the defect location change trend data, the least squares method is used to accurately correct the defect location and generate accurate defect location data.

[0012] Preferably, the step of verifying the calibration results based on precise defect location data, adjusting the deviation portion, and obtaining the defect calibration result dataset includes: Based on accurate defect location data, the defect area is first calibrated. Then, the calibration result is compared with the corresponding defect features in the image using a feature matching method. The matching degree between the calibration result and the actual defect features is calculated, and calibration verification result data is generated. Based on the calibration and verification results, the degree of deviation between the calibration results and the predetermined standard is determined. If the deviation exceeds the set threshold, further correction is performed through the position adjustment algorithm to obtain the corrected defect position data. Based on the corrected defect location data, the calibration results are re-verified, and by further comparing the matching degree between the actual defect characteristics and the calibration results, a defect calibration result dataset is finally generated.

[0013] Preferably, the matching degree between the calculated calibration result and the actual defect characteristics is specifically calculated using cosine similarity.

[0014] The present invention also discloses an image recognition-based bridge defect detection system for performing the method described above, the system comprising: Image acquisition module: Utilizes a high-resolution camera combined with image sensor technology to acquire image data of the bridge surface and generate an image patch dataset; Image patch processing module: Based on the image patch dataset, it uses gradient calculation to perform preliminary analysis, identifies regions in the image with high contrast and / or significant edge changes, performs multi-scale segmentation, optimizes the pixel arrangement within the image patch, and generates an optimized image patch dataset. Region feature extraction module: Extracts local features from image patches, obtains edge intensity, texture features and brightness contrast information, analyzes the correlation between local features and surrounding areas, and generates a region feature correlation matrix.

[0015] Defect location module: Based on the regional feature correlation matrix, it initially determines the location of the defect area, performs position correction based on morphological features and image spatial relationship, dynamically adjusts the defect position, and generates accurate defect location data; Defect calibration module: Based on accurate defect location data, the defect area is calibrated, and the matching degree between the calibration result and the defect features in the image is verified; if there is a deviation, it is corrected through adjustment steps, and a defect calibration result dataset is generated.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention significantly improves the accuracy and efficiency of bridge defect detection by combining multi-scale image processing and adaptive reconstruction algorithms. The acquisition and processing of image patch datasets, through gradient calculation and analysis, effectively identifies regions with high contrast and significant edge changes in the image, thereby accurately locating potential defects. This meticulous feature analysis makes defect region extraction more precise, reducing the probability of false positives and false negatives. Further multi-scale partitioning ensures detailed observation of defect regions at different scales, enabling the identification and optimization of defects over a wider range. The application of the adaptive reconstruction algorithm, by optimizing pixel arrangement, further improves the quality of image patches, enhancing the accuracy of local feature extraction. By meticulously analyzing the correlation between local features and surrounding areas, the innovative scheme can dynamically adjust the defect position in the regional feature correlation matrix. This process greatly improves the accuracy of defect labeling and avoids misjudgments caused by incomplete features or noise interference in traditional methods. Finally, the combination of precise labeling and real-time verification greatly optimizes the labeling results, ensuring the reliability and practicality of defect location data. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation steps of the present invention; Figure 2 This is a schematic diagram illustrating the process of generating a multi-scale reconstructed image patch dataset according to the present invention. Figure 3 This is a flowchart illustrating the generation of a regional feature correlation matrix in some embodiments of the present invention; Figure 4 This is a flowchart illustrating the process of obtaining a defect calibration result dataset in some embodiments of the present invention. Detailed Implementation

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the linguistic context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0019] It should be understood that the terms "system," "device," "unit," and / or "module" as used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0020] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0022] The image recognition-based bridge defect detection method and system provided in the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0023] Figure 1 This is an exemplary flowchart of an image recognition-based bridge defect detection method according to some embodiments of this specification. In some embodiments, the image recognition-based bridge defect detection method can be executed by processing logic, which may include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, etc.), software (instructions running on a processing device to execute hardware simulations), and any combination thereof. In some embodiments, Figure 1 One or more operations in the flowchart of the image recognition-based bridge defect detection method shown can be implemented by a processing device and / or a terminal device. For example, the image recognition-based bridge defect detection method can be stored in a storage device in the form of a computer program and / or instructions, and invoked and / or executed by the processing device and / or the terminal device.

[0024] Specifically, such as Figure 1 As shown, this invention discloses a bridge defect detection method based on image recognition, which specifically includes the following steps: Step 001: Take pictures of the bridge structure to be inspected to obtain image data of the bridge structure surface and obtain image patch dataset; Step 002: Analyze the image patch dataset through gradient calculation, identify regions in the image with high contrast and / or significant edge changes, and divide them into multiple scales based on the regional features. Apply an adaptive reconstruction algorithm to each image patch to optimize the pixel arrangement and obtain a multi-scale reconstructed image patch dataset. Step 003: Extract local features from each image patch in the multi-scale reconstructed image patch dataset, analyze the correlation between local features and surrounding areas, and generate a region feature correlation matrix; the local features include edge intensity, texture features, and brightness contrast; Step 004: Based on the regional feature correlation matrix, determine the preliminary location of the defect, and dynamically adjust the defect calibration through feature correlation analysis to obtain accurate defect location data; Step 005: Based on the accurate defect location data, verify the calibration results, adjust the deviation, and obtain the defect calibration result dataset.

[0025] In a specific embodiment of the present invention, step 001 can be performed by the image acquisition module described below.

[0026] Imaging the bridge structure and acquiring an image patch dataset is the first step in the entire process. This step is fundamental to the bridge defect detection system and involves image acquisition, data preprocessing, and image patch generation. To ensure the quality and accuracy of the image data, the imaging equipment needs to meet certain standards: Image acquisition and equipment selection: First, the choice of acquisition equipment is crucial to ensuring that the acquired image data has sufficient detail and accuracy. Typically, high-resolution cameras are used for image acquisition. These cameras can be DSLR cameras, industrial cameras, or high-resolution surveillance cameras. These devices can capture sufficiently subtle structural changes to ensure that no potential defects are missed during subsequent inspection.

[0027] Furthermore, the choice of image sensor is equally important. Modern cameras commonly use CMOS or CCD sensors. CMOS sensors perform better in low-light conditions, while CCD sensors are better suited for high contrast and detail capture. Therefore, the choice of sensor depends on the shooting environment. To ensure even lighting in the image, adjustable LED light sources or laser scanning systems are often used, especially in poor lighting conditions.

[0028] When acquiring images, the choice of shooting angle is crucial. It's typically necessary to photograph the bridge from multiple angles to comprehensively cover its structure, especially areas with potential defects. For example, drones can be used for high-altitude photography, or stable, adjustable tripods can be used for low-altitude photography to ensure stability and coverage. Regarding image resolution, the acquired images usually need to reach at least 300 dpi (dots per inch) to ensure that the details of defects are clearly visible.

[0029] Generation of image patch datasets: After image acquisition, the next step is to generate the image patch dataset. Because bridges have complex structures and multiple distinct feature regions, dividing the entire image into multiple image patches is a common image processing method. These image patches represent local regions within the entire image.

[0030] Image patch generation is based on the regional features of the image. During processing, by calculating the image gradient, regions with high contrast or significant edge changes can be identified. These regions typically correspond to defects in bridge structures, such as cracks, corrosion, or other surface damage. Therefore, image patch division must ensure accurate capture of these high-contrast regions, especially defective areas.

[0031] When dividing an image into patches, the size is dynamically adjusted based on the region's features. Patches containing high-contrast areas are typically smaller to capture defect features more precisely. Background areas or areas without obvious features are larger to avoid excessive computational burden. Each patch contains the pixel values ​​of that local region, which can be grayscale or RGB values, depending on the image's color mode. Furthermore, each patch also includes gradient information for that region, which aids in subsequent edge and other structural changes detection.

[0032] Image preprocessing: After obtaining the image patch dataset, the images often need to undergo certain preprocessing to further improve image quality and reduce interference. The main purpose of this step is to eliminate noise in the image and improve the identifiability of defective areas.

[0033] Common image preprocessing methods include noise reduction. Images may be affected by different types of noise during the shooting process, impacting image quality. Gaussian filtering is a commonly used method for noise removal, as it smooths the image and eliminates random noise. Median filtering is also a common noise reduction method, particularly suitable for removing irregular noise such as "salt and pepper noise."

[0034] Besides noise reduction, adjusting the brightness and contrast of the image is also a very important preprocessing step. This process can enhance the visibility of defects in the image, especially under uneven lighting conditions. By appropriately adjusting the brightness and contrast, defect areas can be made more prominent, facilitating subsequent detection and analysis.

[0035] Sometimes, image preprocessing also includes color space conversion, especially when it is necessary to convert a color image to a grayscale image. This conversion helps simplify subsequent processing because grayscale images reduce computational load and allow focus on the structural information of the image rather than its color information.

[0036] In addition, edge enhancement is a common step in the processing. By using methods such as the Sobel operator or Canny edge detection, edge information in the image can be enhanced, which is crucial for the identification of defects such as cracks, peeling, and corrosion.

[0037] Output: Image patch dataset After preprocessing, the final image will be divided into multiple image patches, each representing a local region within the image. Each image patch contains not only the original pixel data but also local feature information of that region, such as edge intensity, texture features, and brightness contrast. Furthermore, each image patch also contains gradient information of the image to capture edge or structural changes.

[0038] These image patch datasets will serve as the basis for subsequent analysis and defect detection. They will be used to extract image features, perform pattern recognition, and even accurately locate and label defects.

[0039] like Figure 2 As shown, in another embodiment of the present invention, step 002 is performed by the image block processing module disclosed below, and specifically includes the following sub-steps: Sub-step 0021: Analyze the image patch dataset by calculating the gradient of the Laplacian operator to identify regions in the image with high contrast and / or significant edge changes; The Laplacian operator is a second-order derivative operator commonly used for edge detection and image sharpening. It can detect rapidly changing regions in an image, recognizing not only horizontal and vertical edges but also corners and patches.

[0040] In this sub-step, the gradient information of each image block in the image block dataset needs to be calculated first. Image feature analysis is then performed on each image block to identify areas with high contrast and / or significant edge changes. These areas are often related to bridge defects. The Laplacian operator can effectively capture parts of the image with more dramatic edge changes by calculating the second derivative of the image gray value, thereby filtering out areas containing cracks, corrosion or other surface damage.

[0041] Specifically, for each image patch, its grayscale gradient is first calculated and then processed using the Laplacian operator. The formula for the Laplacian operator is: in, Represents the grayscale value of the image. and These represent the second derivatives of the image along the x-axis and y-axis, respectively. These values ​​measure the rate of change of the image in the horizontal and vertical directions, thus revealing edge information. During the calculation, regions with larger gradient values ​​usually correspond to parts with high contrast or obvious structural changes. Further analysis of these regions can identify potential defect areas.

[0042] For example, in images taken on the surface of a bridge, the grayscale gradient at the location of a crack will show large fluctuations. After processing with the Laplacian operator, the gradient of the image block in the crack area will be significantly higher than that in the background area, thus identifying the crack.

[0043] Sub-step 0022: For image patches containing high-contrast regions, dynamically calculate the optimal size of the image patch based on the edge strength and feature density of the region, set a maximum image patch size limit, and use smaller image patch divisions; for background regions, set a minimum image patch size limit, and use larger image patch divisions to obtain a multi-scale segmented image dataset. When processing image patches containing high-contrast regions, the size of the image patch first needs to be dynamically calculated. The size of the image patch is dynamically adjusted based on the edge strength and feature density of the region. Specifically, regions with strong edge strength indicate complex details and contain more useful defect features; therefore, smaller image patches are needed for refined processing to ensure high-precision capture of defect information. In this case, by calculating the gradient strength of the edges in the image patch, for example using the ΔI(x,y) value obtained using the Laplacian operator, if the gradient value is greater than a certain set threshold (e.g., ΔI(x,y) > 0.5), the region is considered a high-contrast region, and the image patch size should be set smaller to capture more detailed local information. Meanwhile, for background regions, which have fewer variations and less detail, larger image patches are used. Minimum and maximum image patch size limits are set to ensure that the image patch size is dynamically adjusted according to the region characteristics.

[0044] For example, in the background area, assuming the minimum size is set to 32×32 pixels and the maximum size to 128×128 pixels, the image block will be divided into larger areas when the edge strength is weak; while in the defect area, the image block may be divided into smaller blocks of 16×16 pixels to ensure that tiny details of cracks or corrosion can be captured.

[0045] Through such dynamic segmentation, the resulting image dataset can more accurately reflect the characteristics of different areas on the bridge surface, providing strong support for subsequent defect identification.

[0046] Sub-step 0023: After obtaining the multi-scale segmented image dataset, calculate the local features of each image block, identify the background region and / or defect region, use local mean filtering to adjust the weight of each pixel for the defect region to enhance the defect features, use smoothing processing to reduce noise interference for the background region, rearrange the pixels and output the reconstructed image block to obtain the multi-scale reconstructed image block dataset.

[0047] After obtaining the image dataset after multi-scale segmentation, the next step is to perform local feature calculations on each image patch and further analyze whether each image patch contains defective regions.

[0048] First, for each image patch, its local features are calculated, including edge intensity, texture features, and brightness contrast. These features help identify whether the image patch contains defective regions.

[0049] For example, by analyzing the edge information of image patches, potential crack areas can be identified. Areas with higher edge intensity typically represent cracks or fractures, while areas with higher brightness contrast may be related to corrosion areas. After feature extraction, local mean filtering is used to weight each pixel, assigning higher weights to pixels within defect areas to enhance defect features. Specifically, the formula for local mean filtering is: in, These are the filtered image pixel values. It is the original value of the pixel in the neighborhood. This represents the number of neighboring pixels.

[0050] For the background region, smoothing is used to reduce noise interference, and the data structure of the image patch is optimized by rearranging the pixels. The smoothing process mainly involves weighted averaging of each pixel in the image patch to reduce the impact of noise on the image, especially in the background region, where changes are usually small and noise interference is easily absorbed. One commonly used smoothing method is Gaussian filtering. Its basic idea is to assign different weights to neighboring pixels, with weights closer to the center pixel being larger and those farther away being smaller, thus achieving noise reduction and smoothing effects. The formula for Gaussian filtering is as follows: in, This represents the pixel value after filtering. It is located in the image patch The pixel value of the location, These are the weights of the Gaussian kernel function. This refers to the size of the filtering window. The weights of the Gaussian kernel function. According to standard deviation The larger the standard deviation, the wider the weight distribution, and the more pronounced the smoothing effect. This weighted averaging method smooths out noise and subtle fluctuations in the image, resulting in a smoother background area and reducing noise interference in subsequent processing. For example, in the background area of ​​a bridge structure image, weighted averaging of neighboring pixel values ​​effectively reduces minor fluctuations caused by changes in lighting or shooting angle, making the background area more uniform and stable, and helping to better identify and process defective areas.

[0051] Through this processing, the defect information in the image patch is further enhanced, while background noise is effectively suppressed. Ultimately, after these processes, the output multi-scale reconstructed image patch dataset can more clearly represent the defect region, thus providing a more accurate data foundation for subsequent defect identification and localization.

[0052] like Figure 3 As shown, in some embodiments of the present invention, step 003 is specifically performed by the region feature extraction module disclosed below, and specifically includes: Sub-step 0031: Calculate the edge intensity of each image block, extract its texture features and brightness contrast information, and then perform zero-mean unit variance standardization to obtain a local feature dataset; Specifically, when calculating the edge intensity of each image patch, it is first necessary to obtain the gradient information of each patch. This is usually achieved by calculating the first derivatives of the image along the x and y axes, commonly using the Sobel operator. Specifically, the Sobel operator performs a convolution operation on the image, calculating the gradient using the following convolution kernel: By performing a convolution operation on the image, the gradients in the x and y directions of each image patch are obtained, representing the intensity of changes in the image in the horizontal and vertical directions, respectively. Then, a comprehensive index of edge strength is calculated. in, Indicates edge strength. and These are the gradient values ​​in the horizontal and vertical directions, respectively. By calculating the edge intensity of each image patch, it is possible to identify which regions exhibit significant structural changes; these regions are often potential defect areas. For example, suppose in an image patch of a bridge, the calculated edge intensity value is 50, while the edge intensity of the background region is 10. Then, the higher edge intensity in this image patch indicates that it may contain defects, such as cracks.

[0053] Next, texture features and brightness contrast information are extracted. Texture features can be obtained by performing statistical analysis of local grayscale values ​​in the image, for example, using the Gray-Level Co-occurrence Matrix (GLCM) to calculate texture features. Specifically, the joint probability distribution of pixel pairs in the image can be calculated, and indices such as contrast, correlation, and energy can be calculated using this distribution. Brightness contrast is obtained by calculating the brightness differences in local regions; for example, the standard deviation of the region can be used as a quantification of brightness contrast. These texture and brightness contrast features help further distinguish defective areas from background areas. After feature extraction, the data needs to be standardized with zero mean and unit variance to ensure that the mean of each feature is 0 and the standard deviation is 1, avoiding the unbalanced effects caused by differences in feature values. The standardization formula is: in, These are the original eigenvalues. It is the mean of the features. It is the standard deviation of the feature. Standardization can eliminate scale differences between different features, ensuring that each feature plays an equal role in subsequent processing.

[0054] Sub-step 0032: Based on the local feature dataset, Gaussian filter is used to perform spatial filtering on each image patch, Canny operator edge detection is used to identify significant edges in the image, and K-means clustering is used to analyze the spatial correlation between local features in the image patch and the surrounding area to obtain the optimized feature dataset. The main function of Gaussian filtering is to smooth an image by weighting neighboring pixels, reducing noise, and preserving the smoothness of edge features. The weights of the Gaussian filter are determined by a Gaussian function, as shown in the formula: in, These are the weights of the Gaussian filter. The standard deviation determines the range of the filter. In this way, the pixel value in each image block is weighted according to the values ​​of its neighboring pixels, thus smoothing the image and reducing the impact of noise. Especially in background areas, this processing can effectively remove irrelevant background noise, making the image cleaner and highlighting defect features. For example, in the background area of ​​a bridge surface image, after Gaussian filtering, noise caused by changes in lighting and shooting angle will be smoothed out, thereby enhancing the crack features in the image.

[0055] Meanwhile, the Canny operator is used for edge detection. The Canny operator effectively identifies significant edges in an image by calculating the image gradient and utilizing non-maximum suppression and double thresholding. The calculation steps of the Canny operator are as follows: first, the image gradient is calculated; then, non-maximum suppression is performed to remove insignificant edges; finally, double thresholding is performed to determine the intensity and direction of the edges, ultimately extracting significant edges in the image to help further identify defect areas.

[0056] Then, the K-means clustering algorithm is used to analyze the spatial correlation between local features in the image patch and the surrounding area. The K-means clustering algorithm process includes initializing K cluster centers, then iteratively calculating the distance from each data point to each cluster center, assigning the data point to the nearest cluster center, and updating the cluster centers until convergence.

[0057] K-means clustering can be used to classify local features in an image patch into different categories, thereby identifying different region types, such as defect regions and background regions.

[0058] For example, suppose that in the clustered feature dataset, the feature values ​​of certain regions are highly concentrated, indicating that these regions are highly correlated with defect features.

[0059] Sub-step 0033: Based on the optimized feature dataset, match the local features of each image block with the preset standard template, calculate their feature similarity, and select the features most relevant to the defect area. Finally, identify the defect area and generate a region feature association matrix.

[0060] The standard template includes edge features, texture features, brightness contrast features, color features, and defect-specific template features. Defect-specific template features include crack template features, erosion template features, and peeling template features. By calculating the similarity between each image patch and the template, the features most relevant to the defect region can be selected. Cosine similarity can be used to calculate the similarity, with the formula: in, and These are the image patch feature vector and the standard template feature vector, respectively. and These are the moduli of the feature vectors. By calculating the similarity, image patches with the highest similarity to the standard template can be selected, and these image patches are most likely to contain defective regions.

[0061] For example, suppose the similarity between the features of an image patch and a crack template is 0.85. This means that the image patch is very similar to the crack features, so it can be considered that the image patch contains a crack. Finally, the generated region feature association matrix will contain the matching results of all image patches with different defect templates, which will be used for subsequent defect localization and labeling.

[0062] Here is an example of a specific region feature correlation matrix: Suppose we have an image patch dataset containing 5 image patches, each with three features: edge intensity, texture features, and brightness contrast. After standardization, the features of each image patch yield the following local feature vectors (edge ​​intensity, texture features, brightness contrast): Step 1: Calculate similarity: Based on the feature vectors in the table above, we calculate the feature similarity between each pair of image patches. One commonly used metric is cosine similarity. The formula for cosine similarity is: ; in, and These are the feature vectors of two image patches, respectively. and It is their model.

[0063] Taking image patches A and B as an example, we calculate their cosine similarity. First, we calculate the dot product of their eigenvectors: Then, calculate the modulo of A and B: ; Finally, calculate the cosine similarity: ; Similarly, the similarity of other image patch pairs can be calculated.

[0064] Step 2: Construct the regional feature association matrix: By calculating the cosine similarity of all image patch pairs, we can obtain a region feature correlation matrix, as shown below: In this matrix, each value represents the similarity between two image patches. For example, image patches A and B have a similarity of 0.998, indicating that their features are very similar; image patches A and C have a similarity of 0.95, indicating that they have some similarity, but it is slightly weaker than that between A and B.

[0065] Step 3: Apply the matrix This matrix can be used for further analysis, such as identifying which image patches share highly similar features, suggesting they may belong to the same defect region, or determining whether an image patch belongs to the background region. By applying a threshold to this matrix, the preliminary location of the defect region can be identified; for example, selecting image patches with a similarity greater than 0.90 for association can ultimately help determine the location and size of the defect region.

[0066] In some embodiments of the present invention, step 004 is specifically executed by the defect location module described below, and specifically includes the following sub-steps: Sub-step 0041: Based on the regional feature correlation matrix, extract the edge features and texture features of the defect region, and preliminarily determine the defect location and orientation through image segmentation and contour matching methods to generate preliminary defect location data; For example, in the analysis based on the region feature correlation matrix, the edge features and texture features of the defect region are first extracted.

[0067] Edge features can be obtained through gradient calculation. Specifically, the gradient value of each pixel in the image is calculated, and the image is processed using the Laplacian operator to obtain change information in high-frequency regions and detect edges. The specific method of processing the image using the Laplacian operator has been described in the aforementioned sub-step 0021. Texture features can be extracted using the gray-level co-occurrence matrix (GLCM), which will help analyze the texture features of different regions in the image, such as changes in roughness or detail texture.

[0068] Next, image segmentation techniques are used to divide the image into multiple regions. K-means clustering is employed to segment the image into background and potential defect regions. Then, contour matching is used to analyze the morphological features of the segmented regions, and edge detection algorithms (such as the Canny operator) are used to identify significant edges in the image, which helps in identifying defect regions. By comparing the segmented regions with pre-defined defect feature templates, the location and orientation of the defects can be preliminarily determined. For example, suppose an image of a bridge shows significant gradient changes and obvious texture variations in the crack region. In this case, edge detection can effectively identify the crack region, thereby generating preliminary defect location data. Sub-step 0042: Based on the preliminary defect location data, use similarity analysis to calculate the defect location change trend, and combine the spatial relationship of adjacent image blocks to dynamically adjust the defect location calibration to obtain defect location change trend data; Similarity analysis is performed by comparing the feature vectors (e.g., edge features, texture features, and brightness contrast) of the current image patch with those of neighboring image patches. The cosine similarity formula, which was explained in sub-step 0032, is used and will not be repeated here. A higher similarity value indicates a higher similarity between image patches. Then, optical flow is used to calculate the positional change trend of the defect region. Optical flow determines the positional change of an image patch across different frames by tracking its motion. The algorithm calculates the displacement of the defect region relative to neighboring image patches, thus obtaining the trend of defect position change. By combining the spatial relationships of neighboring image patches, the defect position calibration can be dynamically adjusted.

[0069] For example, when detecting cracks in a section of a bridge, assuming the similarity of the image blocks is 0.85 and the displacement calculated by the optical flow method is 5 pixels, it means that the defect area has been displaced relative to the previous image block, and the defect position can be adjusted next.

[0070] Sub-step 0043: Based on the defect location change trend data, use the least squares method to accurately correct the defect location and generate accurate defect location data.

[0071] After acquiring the defect location change trend data, the defect location is precisely corrected using the least squares method. The least squares method is used to fit the defect location change trend curve; by substituting the defect location change trend data of all image blocks, the optimal fitting parameters are calculated. Assuming the defect location change trend data is... Each This represents the displacement of the defect location within the i-th image block. The least squares method corrects the location by minimizing the sum of squared errors; the specific calculation formula is as follows: ; in, The actual value of the i-th data point. Based on fitted parameters The predicted value is represented by the sum of squared errors, which is the total error.

[0072] By minimizing the error, the precise location of the defect can be obtained. Assuming the fitted location error is 2 pixels, the corrected defect location data can then be obtained.

[0073] Please see Figure 4 In another embodiment of the present invention, step 005 is specifically executed by the defect identification module described below, and specifically includes: Sub-step 0051: Based on the accurate defect location data, the defect area is first calibrated. The calibration result is compared with the corresponding defect features in the image using the feature matching method. The matching degree between the calibration result and the actual defect features is calculated, and calibration verification result data is generated. When calibrating based on precise defect location data, the first step is to compare the calibration results of the defect area with the corresponding defect features in the image. The core of the calibration process is to determine the specific location of the defect through feature matching technology. The feature matching method extracts representative feature points from the image, such as crack edge features and texture features, and compares them with a pre-set standard template to ensure that the identified defect area is consistent with the defect area in the actual image. In the feature matching process, cosine similarity is widely used in calculating the matching degree, and the cosine similarity formula has been explained in the aforementioned sub-step 0032.

[0074] The cosine similarity score ranges from 0 to 1, with a higher value indicating greater similarity between the two feature vectors. In this step, assuming feature matching is performed on a crack in a bridge image, the calculated cosine similarity is 0.92, indicating a high degree of consistency in the feature matching results. Therefore, the calibration verification results can be obtained, confirming a high degree of match between the calibration results and the defect area in the image.

[0075] Sub-step 0052: Based on the calibration verification result data, determine the degree of deviation between the calibration result and the predetermined standard. If the deviation exceeds the set threshold, further correction is performed through the position adjustment algorithm to obtain the corrected defect position data. Based on the calibration and verification results, the next task is to determine the degree of deviation between the calibration results and the predetermined standard. In this process, a deviation threshold is set, for example, 5 pixels. If the deviation of the calibration results from the predetermined standard exceeds this threshold, further correction is required. Deviation calculation can be performed by comparing the distance between the calibration position and the actual defect position, using the Euclidean distance formula: in, The coordinates of the calibrated position, Let D be the coordinates of the actual defect location, and D be the calculated deviation value. In this example, if the calibration result is (100, 150) and the actual defect location is (103, 148), the calculated Euclidean distance is 3.16 pixels, which does not exceed the set deviation threshold of 5 pixels, so no further correction is needed. However, if the deviation exceeds this threshold, for example, if the calculated D is 6 pixels, then the defect location needs to be further corrected using a position adjustment algorithm to ensure a more accurate calibration position.

[0076] The most common and widely used position adjustment algorithm is the translation adjustment method based on the least squares method. It is a common technical means in the existing technology, so we will not elaborate on it.

[0077] Sub-step 0053: Based on the corrected defect location data, re-verify the calibration results, and finally generate a defect calibration result dataset by further comparing the matching degree between the actual defect features and the calibration results.

[0078] After adjusting the location, the next step is to re-verify the calibration results based on the corrected defect location data. During re-verification, the effectiveness of the correction is confirmed by comparing the matching degree between the calibration results and the actual defect features. First, the cosine similarity method is used again to perform feature matching on the corrected defect location, and the matching degree is calculated again. If the cosine similarity value is high after correction, it indicates that the calibration result is accurate. For example, if the recalculated cosine similarity is 0.98 after correction, it means that the corrected defect location matches the defect region in the actual image more closely. Ultimately, this process generates an accurate defect calibration result dataset, providing reliable data for subsequent defect detection and evaluation.

[0079] Based on the above method, another embodiment of the present invention discloses a bridge defect detection system based on image recognition. The system includes: an image acquisition module: using a high-resolution camera combined with image sensor technology to acquire bridge surface image data and generate an image block dataset; Image patch processing module: Based on the image patch dataset, it uses gradient calculation to perform preliminary analysis, identifies regions in the image with high contrast and / or significant edge changes, performs multi-scale segmentation, optimizes the pixel arrangement within the image patch, and generates an optimized image patch dataset. Region feature extraction module: Extracts local features from image patches, obtains edge intensity, texture features and brightness contrast information, analyzes the correlation between local features and surrounding areas, and generates a region feature correlation matrix.

[0080] Defect location module: Based on the regional feature correlation matrix, it initially determines the location of the defect area, performs position correction based on morphological features and image spatial relationship, dynamically adjusts the defect position, and generates accurate defect location data; Defect calibration module: Based on accurate defect location data, the defect area is calibrated, and the matching degree between the calibration result and the defect features in the image is verified; if there is a deviation, it is corrected through adjustment steps, and a defect calibration result dataset is generated.

[0081] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0082] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0083] Furthermore, those skilled in the art will understand that various aspects of this specification can be described and illustrated in several patentable ways or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, various aspects of this specification can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. All of the above hardware or software may be referred to as a “data block,” “module,” “engine,” “unit,” “component,” or “system.” Furthermore, various aspects of this specification may be represented as a computer product located on one or more computer-readable media, including computer-readable program code.

[0084] Computer storage media may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and suitable combinations thereof. Computer storage media can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0085] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc.; conventional procedural programming languages ​​such as C, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or processing device. In the latter case, the remote computer can be connected to the user's computer through any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0086] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented by hardware devices, they can also be implemented solely by software solutions, such as installing the described system on existing processing devices or mobile devices.

[0087] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0088] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A bridge defect detection method based on image recognition, characterized in that, The method includes: The bridge structure to be inspected is photographed to obtain surface image data of the bridge structure and to obtain an image patch dataset. By analyzing the image patch dataset through gradient calculation, regions with high contrast and / or significant edge changes in the image are identified. Based on the regional characteristics, the images are divided into multiple scales. An adaptive reconstruction algorithm is applied to each image patch to optimize the pixel arrangement, resulting in a multi-scale reconstructed image patch dataset. Local features are extracted from each image patch in the multi-scale reconstructed image patch dataset, and the correlation between local features and surrounding areas is analyzed to generate a region feature correlation matrix; the local features include edge intensity, texture features, and brightness contrast. Based on the regional feature correlation matrix, the initial location of the defect is determined, and the defect calibration is dynamically adjusted through feature correlation analysis to obtain accurate defect location data; Based on accurate defect location data, the calibration results are verified, and the deviations are adjusted to obtain the defect calibration result dataset.

2. The bridge defect detection method based on image recognition according to claim 1, characterized in that: The gradient algorithm is specifically the Laplacian operator; The multi-scale segmentation based on regional features specifically involves: for image patches containing high-contrast regions, dynamically calculating the optimal size of the image patch based on the edge strength and feature density of the region, setting a maximum image patch size limit, and using smaller image patch segmentation; for background regions, setting a minimum image patch size limit, and using larger image patch segmentation to obtain a multi-scale segmented image dataset.

3. The bridge defect detection method based on image recognition according to claim 2, characterized in that, An adaptive reconstruction algorithm is applied to each image patch to optimize the pixel arrangement, resulting in a multi-scale reconstructed image patch dataset, specifically: After obtaining the multi-scale segmented image dataset, local features are calculated for each image patch. Identify background and / or defect regions. For defect regions, use local mean filtering to adjust the weight of each pixel to enhance defect features. For background regions, use smoothing to reduce noise interference. Rearrange the pixels and output the reconstructed image patch to obtain the multi-scale reconstructed image patch dataset.

4. The bridge defect detection method based on image recognition according to claim 1, characterized in that, The process of extracting local features from each image patch in the multi-scale reconstructed image patch dataset, analyzing the correlation between local features and surrounding areas, and generating a region feature correlation matrix includes: The edge intensity of each image patch is calculated, its texture features and brightness contrast information are extracted, and then zero-mean unit variance standardization is performed to obtain the local feature dataset. Based on the local feature dataset, Gaussian filters are used to perform spatial filtering on each image patch, Canny edge detection is used to identify significant edges in the image, and K-means clustering is used to analyze the spatial correlation between local features in the image patch and the surrounding area to obtain the optimized feature dataset. Based on the optimized feature dataset, the local features of each image patch are matched with a preset standard template, their feature similarity is calculated, and the features most relevant to the defect area are selected. Finally, the defect area is identified, and a regional feature association matrix is ​​generated.

5. The bridge defect detection method based on image recognition according to claim 4, characterized in that, The standard template includes edge features, texture features, brightness contrast features, color features, and defect-specific template features, including crack template features, corrosion template features, and peeling template features.

6. The bridge defect detection method based on image recognition according to claim 1, characterized in that, The process of determining the preliminary defect location based on the regional feature correlation matrix, and dynamically adjusting the defect calibration through feature correlation analysis to obtain precise defect location data includes: Based on the regional feature correlation matrix, the edge features and texture features of the defect area are extracted. Through image segmentation and contour matching methods, the location and orientation of the defect are initially determined, and preliminary defect location data is generated. Based on the preliminary defect location data, similarity analysis and optical flow method are used to calculate the defect location change trend. Combined with the spatial relationship of adjacent image blocks, the defect location calibration is dynamically adjusted to obtain defect location change trend data. Based on the defect location change trend data, the least squares method is used to accurately correct the defect location and generate accurate defect location data.

7. The bridge defect detection method based on image recognition according to claim 1, characterized in that, The process involves verifying the calibration results based on precise defect location data, adjusting for deviations, and obtaining a defect calibration result dataset, including: Based on accurate defect location data, the defect area is first calibrated. Then, the calibration result is compared with the corresponding defect features in the image using a feature matching method. The matching degree between the calibration result and the actual defect features is calculated, and calibration verification result data is generated. Based on the calibration and verification results, the degree of deviation between the calibration results and the predetermined standard is determined. If the deviation exceeds the set threshold, further correction is performed through the position adjustment algorithm to obtain the corrected defect position data. Based on the corrected defect location data, the calibration results are re-verified, and by further comparing the matching degree between the actual defect characteristics and the calibration results, a defect calibration result dataset is finally generated.

8. The bridge defect detection method based on image recognition according to claim 7, characterized in that, The matching degree between the calculated calibration results and the actual defect characteristics is specifically calculated using cosine similarity.

9. A bridge defect detection system based on image recognition, used to perform the method according to any one of claims 1-8, characterized in that, The system includes: Image acquisition module: Utilizes a high-resolution camera combined with image sensor technology to acquire image data of the bridge surface and generate an image patch dataset; Image patch processing module: Based on the image patch dataset, it uses gradient calculation to perform preliminary analysis, identifies regions in the image with high contrast and / or significant edge changes, performs multi-scale segmentation, optimizes the pixel arrangement within the image patch, and generates an optimized image patch dataset. Region feature extraction module: Extracts local features from image patches, obtains edge intensity, texture features and brightness contrast information, analyzes the correlation between local features and surrounding areas, and generates a region feature correlation matrix. Defect location module: Based on the regional feature correlation matrix, it initially determines the location of the defect area, performs position correction based on morphological features and image spatial relationship, dynamically adjusts the defect position, and generates accurate defect location data; Defect calibration module: Based on accurate defect location data, the defect area is calibrated, and the matching degree between the calibration result and the defect features in the image is verified; if there is a deviation, it is corrected through adjustment steps, and a defect calibration result dataset is generated.

Images