Crack detection and pseudo labeling method and system based on deep learning and genetic algorithm

By combining deep learning and multi-objective genetic algorithms, the pseudo labeling process is optimized, and the problem of high labeling cost of crack detection data in hydraulic buildings is solved, high-precision and high-reliability crack detection is achieved, and the safety of the building is improved .

CN119048522AActive Publication Date: 2024-11-29SANXIA JINSHAJIANG YUNCHUAN HYDROPOWER DEV CO LTD

Patent Information

Application Number
CN202411556692.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-11-29
Estimated Expiration
2044-11-04

AI Technical Summary

Technical Problem

Existing deep learning methods require a large amount of high-quality labeled data in crack detection, and the cost of obtaining these data is high, making it difficult to effectively solve in large-scale hydraulic building inspections, affecting the high accuracy and reliability of the inspection.

Method used

Combining deep learning and multi-objective genetic algorithms, crack positioning is performed through Grad-CAM algorithm, multi-objective genetic algorithm is introduced to optimize the parameters of semantic segmentation module, generate high-quality pseudo-notation data, reduce manual intervention, and improve the accuracy and coverage of pseudo-notation Rate.

Benefits of technology

It effectively reduces the cost of data labeling, improves the accuracy and reliability of crack detection, reduces the risks of missed and missed inspections, and improves the safety monitoring and maintenance capabilities of hydraulic buildings.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048522A_ABST
    Figure CN119048522A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of crack detection, and discloses a crack detection and false labeling method and system based on deep learning and a genetic algorithm, and the method comprises the steps: collecting and preparing unlabeled original image data; feature learning is carried out on the original image through a deep learning classification model, and a Grad-CAM algorithm is utilized to output a positioning map of cracks in the image; the original image is input into a semantic segmentation module at the same time, and pixel-level crack identification is carried out; in the semantic segmentation module, a multi-target genetic algorithm is introduced to optimize segmentation parameters; and carrying out image fusion on output results of the crack detection and positioning module and the semantic segmentation module, applying morphological closed operation on a fused result image, filling a crack cavity generated by threshold segmentation, and outputting a final crack pseudo-labeling result image. According to the method, the accuracy and the coverage rate of pseudo labeling are improved, the optimal image scale and the optimal data scale in the pseudo labeling process are taken into an optimization target, and the quality and the practicability of pseudo labeling are further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of crack detection technology, and in particular to a crack detection and pseudo-labeling method and system based on deep learning and genetic algorithm. Background Art

[0002] Existing hydraulic structures, especially dams, have long faced a variety of structural problems such as cracks, wear, and erosion, which may pose direct or indirect threats to the stability and safety of the structures. In order to effectively identify and deal with these problems, deep learning methods have been widely adopted in the field of crack detection and have become the mainstream strategy in academia and industry. However, these deep learning methods, especially supervised learning models, usually require a large number of accurately annotated crack images as training data. Obtaining these high-quality training data is not only time-consuming but also costly, as it requires professionals to accurately annotate, which is particularly prominent in large-scale hydraulic structure detection.

[0003] In view of the important role of hydraulic structures in flood prevention and social stability, it is particularly urgent to develop a method for efficiently obtaining high-quality training data. This method needs to ensure high accuracy and reliability of crack detection and segmentation while reducing the cost of data annotation. Therefore, the present invention comes into being, aims to solve the above problems, and proposes an intelligent crack detection and pseudo-annotation generation method combining deep learning and adaptive genetic algorithm, which provides strong technical support for the safety monitoring and maintenance of hydraulic structures. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a crack detection and pseudo-labeling method based on deep learning and genetic algorithm, which can introduce a multi-objective genetic algorithm to automatically optimize the parameters of the threshold segmentation algorithm and improve the accuracy and coverage of pseudo-labeling.

[0006] To solve the above technical problems, the present invention provides the following technical solutions, a crack detection and pseudo-labeling method based on deep learning and genetic algorithm, comprising: collecting and preparing unlabeled original image data as the input of the processing flow; inputting the original image into the crack detection and positioning module, applying the deep learning classification model to perform feature learning, and using the Grad-CAM algorithm to output the positioning map of the cracks in the image; the original image is simultaneously input into the semantic segmentation module, and the deep learning model is used to perform pixel-level crack recognition; in the semantic segmentation module, a multi-objective genetic algorithm is introduced to optimize the segmentation parameters; the output results of the crack detection and positioning module and the semantic segmentation module are image fused, and a morphological closing operation is applied to the fused result image to fill the crack holes generated by the threshold segmentation; after the morphological closing operation, the final crack pseudo-labeling result image is output.

[0007] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, the original image data includes collecting images covering various cracks, wear and erosion structural problems, and under different environmental conditions using a high-resolution camera; at the same time, an on-site survey of the hydraulic structure is conducted to determine the key areas and locations for image acquisition, and during the acquisition process, the metadata of the image is recorded; Key areas and locations include load-bearing support points, beam-column connections, and foundations; they also include sluice gates, valves, spillways with service functions, and areas where historical data shows frequent cracks or damage, and areas that have been repaired or reinforced; The quality of the collected images is evaluated, blurred, overexposed and underexposed images are removed, and the screened images are preprocessed, including denoising, enhancement and correction.

[0008] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, wherein: the crack location map in the output image includes using Grad-CAM to output the crack location heat map to improve the crack location accuracy during the crack detection and splitting process; Pass the image forward through the CNN model to get the predicted score of the category: , in, represents the prediction score of the category, represents the weight of the connection layer, represents the input image, represents the CNN model, Represents the bias of the connection layer; Select the class with the highest prediction score and calculate the gradient of the prediction score with respect to the output of the last convolutional layer: , , in, Indicates the selection category. Indicates category The score, Represents the feature map of the convolutional layer; Perform global average pooling on the gradient values ​​to obtain weights, multiply the weights with the feature map and sum them to obtain the final class activation map: , , in, represents the weight, Representation feature map The dimension of represents the class activation map; The class activation map is upsampled to the size of the input image, generating a heatmap that can be overlaid on the original image.

[0009] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, the crack identification includes: inputting the unlabeled original image into a semantic segmentation module, performing pixel-level crack identification, and generating a segmented crack semantic pixel map; Process the grayscale value of the original image and introduce a normalization function to ensure numerical stability: , , in, Represents the pixel point in the original image ( x , y ), represents the mean value of the image grayscale value, represents the normalization function, Indicates the minimum and maximum values ​​in the image; Using the deep learning segmentation network FCN, we can get the segmented crack semantic pixel map: , in, represents the activation function, Represents a segmentation model.

[0010] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, the optimization of the segmentation parameters includes automatically optimizing the parameters of the threshold segmentation algorithm through a multi-objective genetic algorithm to improve the quality of pseudo-labeling, and defining two optimization goals, the accuracy rate A and the coverage rate C' of the pseudo-labeling. , , , in, represents the optimization objective function, represents the weight factor, represents the threshold segmentation parameter, represents the genetic algorithm control parameter, represents the true label of the i-th data point, represents the label predicted by the threshold segmentation algorithm, represents the total number of data points, represents the indicator function, Represents a special label used to indicate a data point that the algorithm cannot determine.

[0011] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, wherein: the image fusion includes fusing two binary images using a logical operation, and using a logical OR operation to ensure that all parts of the crack are marked; The fusion operation is represented by the following numerical expression: , in, is the fused image, It is the crack location map. is the threshold segmentation image, Represents a logical OR operation.

[0012] As a preferred solution of the crack detection and pseudo-labeling method based on deep learning and genetic algorithm described in the present invention, the morphological closing operation includes dilation and erosion operations, wherein the dilation operation dilates or grows the bright area in the image, and the erosion operation shrinks the bright area; The closing operation first performs an expansion operation and then an erosion operation. The specific expression is as follows: Assume A is the original image, B is the structural element, and the expansion operation and erosion operations They are defined as: , , in, represents the position of the structural element B on image A, represents the displacement of B; Morphological closing operation A ⋅ B Defined as: , After morphological closing operation, the final crack pseudo-annotation result image is output, and the similarity between the image and the real annotation is increased.

[0013] Another object of the present invention is to provide a crack detection and pseudo-labeling system based on deep learning and genetic algorithm, which can incorporate the optimal image scale and the optimal data scale in the pseudo-labeling process into the optimization target, further improving the quality and practicality of the pseudo-labeling.

[0014] As a preferred solution of the crack detection and pseudo-labeling system based on deep learning and genetic algorithm described in the present invention, it includes: a crack detection and positioning module, a semantic segmentation module, a multi-objective genetic algorithm optimization module, and a pixel-by-pixel fusion module; The crack detection and positioning module uses a deep learning classification model to identify and locate cracks in the image; The semantic segmentation module performs pixel-level crack recognition and generates a segmented crack semantic pixel map; The multi-objective genetic algorithm optimization module automatically optimizes the parameters of the threshold segmentation algorithm through the multi-objective genetic algorithm to improve the quality of pseudo-labeling; The pixel-by-pixel fusion module is responsible for fusing the crack splitting and positioning results with the semantic segmentation results, and outputting the final pseudo-annotated image.

[0015] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of a crack detection and pseudo-labeling method based on deep learning and genetic algorithm are implemented.

[0016] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of a crack detection and pseudo-labeling method based on deep learning and genetic algorithm are implemented.

[0017] The beneficial effects of the present invention are as follows: the accuracy and coverage of pseudo-annotation can be effectively improved by automatically optimizing the parameters of the threshold segmentation algorithm through a multi-objective genetic algorithm. Combining deep learning and a multi-objective genetic algorithm, pixel-level pseudo-annotation can be automatically generated without manual intervention. The optimal image scale and the optimal data scale in the pseudo-annotation process are incorporated into the optimization target to ensure that the generated pseudo-annotated images can maintain high quality at different image and data scales. High-quality pseudo-annotated images help improve the performance of the crack detection model and reduce the risk of missed detection and false detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 A schematic flow chart of a crack detection and pseudo-labeling method based on deep learning and genetic algorithm provided in accordance with an embodiment of the present invention.

[0019] Figure 2 A technical architecture diagram of a crack detection and pseudo-labeling method based on deep learning and genetic algorithm provided for one embodiment of the present invention.

[0020] Figure 3A crack location process of a Grad-Cam algorithm for crack detection and pseudo-labeling method based on deep learning and genetic algorithm is provided as an embodiment of the present invention.

[0021] Figure 4 An overall flow chart of a crack detection and pseudo-labeling method based on deep learning and genetic algorithm provided by an embodiment of the present invention, which combines crack positioning and semantic segmentation to obtain a final pseudo-labeling result.

[0022] Figure 5 A schematic diagram of the working modules of a crack detection and pseudo-labeling system based on deep learning and genetic algorithm provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0025] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0026] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0027] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0028] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] Example 1, reference Figure 1-Figure 4 , which is the first embodiment of the present invention, provides a crack detection and pseudo-labeling method based on deep learning and genetic algorithm, including: S1: Collect and prepare unlabeled raw image data as the input of the processing flow, input the raw image into the crack detection and localization module, use the deep learning classification model for feature learning, and use the Grad-CAM algorithm to output the location map of the cracks in the image.

[0030] Furthermore, the raw image data includes, using a high-resolution camera, collecting images covering various cracks, wear and tear, erosion structural problems, and images under different environmental conditions; at the same time, conducting on-site surveys of hydraulic structures to determine key areas and locations for image collection, and recording metadata of the images during the collection process; Key areas and locations include load-bearing support points, beam-column connections, and foundations; they also include sluice gates, valves, spillways with service functions, and areas where historical data shows frequent cracks or damage, and areas that have been repaired or reinforced; The quality of the collected images is evaluated, blurred, overexposed and underexposed images are removed, and the screened images are preprocessed, including denoising, enhancement and correction.

[0031] Furthermore, the crack location map in the output image includes a Grad-CAM output crack location heat map used in the crack detection and splitting process to improve crack location accuracy; Pass the image forward through the CNN model to get the predicted score of the category: , in, represents the prediction score of the category, represents the weight of the connection layer, represents the input image, represents the CNN model, Represents the bias of the connection layer; It should be noted that the class with the highest prediction score is selected and the gradient of the prediction score with respect to the output of the last convolutional layer is calculated: , , in, represents the prediction score of the category, represents the weight of the connection layer, represents the input image, represents the CNN model, Represents the bias of the connection layer; Perform global average pooling on the gradient values ​​to obtain weights, multiply the weights with the feature map and sum them to obtain the final class activation map: , , in, represents the weight, Representation feature map The dimension of represents the class activation map; The class activation map is upsampled to the size of the input image, generating a heatmap that can be overlaid on the original image.

[0032] S2: The original image is simultaneously input into the semantic segmentation module, and the deep learning model is used to perform pixel-level crack recognition. In the semantic segmentation module, a multi-objective genetic algorithm is introduced to optimize the segmentation parameters.

[0033] Furthermore, the crack identification includes inputting the unlabeled original image into a semantic segmentation module, performing pixel-level crack identification, and generating a segmented crack semantic pixel map; Process the grayscale value of the original image and introduce a normalization function to ensure numerical stability: , , in, Represents the pixel point in the original image ( x , y ), represents the mean gray value of the image, represents the normalization function, Indicates the minimum and maximum values ​​in the image; Using the deep learning segmentation network FCN, we can get the segmented crack semantic pixel map: , in, represents the activation function, Represents a segmentation model.

[0034] It should be noted that the optimization of the segmentation parameters includes automatically optimizing the parameters of the threshold segmentation algorithm through a multi-objective genetic algorithm to improve the quality of pseudo-labeling and define two optimization goals, namely, the accuracy rate A and the coverage rate C' of the pseudo-labeling. , , , in, represents the optimization objective function, represents the weight factor, represents the threshold segmentation parameter, represents the genetic algorithm control parameter, represents the true label of the i-th data point, represents the label predicted by the threshold segmentation algorithm, represents the total number of data points, represents the indicator function, Represents a special label used to indicate a data point that the algorithm cannot determine.

[0035] S3: Perform image fusion on the output results of the crack detection and positioning module and the semantic segmentation module, apply morphological closing operation to the fused result image, and fill the crack holes generated by threshold segmentation.

[0036] Furthermore, the image fusion includes fusing the two binary images using a logical operation, using a logical "OR" operation to ensure that all parts of the crack are marked; The fusion operation is represented by the following numerical expression: , in, is the fused image, It is the crack location map. is the threshold segmentation image, Represents a logical OR operation.

[0037] S4: After morphological closing operation, the final crack pseudo-annotation result image is output.

[0038] Furthermore, the morphological closing operation includes dilation and erosion operations, wherein the dilation operation dilates or grows the bright area in the image, and the erosion operation shrinks the bright area; The closing operation first performs an expansion operation and then an erosion operation. The specific expression is as follows: Assume A is the original image, B is the structural element, and the expansion operation and erosion operations They are defined as: , , in, represents the position of the structural element B on image A, represents the displacement of B; Morphological closing operation A ⋅ B Defined as: , After morphological closing operation, the final crack pseudo-annotation result image is output, and the similarity between the image and the real annotation is increased.

[0039] Example 2, an embodiment of the present invention, provides a crack detection and pseudo-labeling method based on deep learning and genetic algorithm. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0040] 1. In the data preparation stage, the raw image data undergoes preprocessing operations such as denoising and contrast enhancement to improve the accuracy of subsequent processing. A dataset containing a variety of industrial inspection images was collected, including 500 images with cracks and 500 images without cracks. The resolution of these images is 1024x1024 pixels, covering various crack morphologies and backgrounds.

[0041] 2. The image data enters two processing routes in parallel: one is the crack detection and location route based on the classification model, and the other is the semantic segmentation route. Three deep convolutional neural network architectures, U-Net, Resnet50 and VGG, are used to train the image classification model. Each model has been iterated and adjusted many times to achieve the best classification performance. During the training process, we applied the Grad-CAM algorithm to assist in locating the cracks. By calculating the gradient value of the image block, the exact location of the crack can be obtained.

[0042] 3. In the crack detection and positioning route, the deep learning model identifies the crack features in the image and locates the crack area. The Grad-CAM algorithm is used to generate a thermal map. Finally, the thermal map and positioning map are combined to obtain the crack location result.

[0043] 4. At the same time, the semantic segmentation route performs pixel-level crack detection through a deep learning model and tunes model parameters through a multi-objective genetic algorithm to balance accuracy and other performance indicators. Finally, this module outputs a pixel-level crack segmentation result map.

[0044] 5. The output results of the two routes are merged, and the accuracy and integrity of the crack area are guaranteed through logical operations. After merging and fusion, the crack image with pseudo annotations is output, and then denoised and enhanced through morphological closing operations to output the final result.

[0045] 6. Apply morphological closing operation to fill the holes caused by threshold segmentation and smooth the edges of cracks.

[0046] 7. Finally, pseudo-annotated images are generated, which can be used for training and testing of deep learning models to verify and improve the performance of crack detection.

[0047] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0048] Embodiment 3, the third embodiment of the present invention, is different from the first two embodiments in that: If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk, etc., which can store program codes.

[0049] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0050] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0051] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0052] Example 4, reference Figure 5 , which is an embodiment of the present invention, provides a crack detection and pseudo-labeling system based on deep learning and genetic algorithm, which is characterized by: including a crack detection and positioning module, a semantic segmentation module, a multi-objective genetic algorithm optimization module, and a pixel-by-pixel fusion module.

[0053] The crack detection and localization module uses a deep learning classification model to identify and locate cracks in images.

[0054] The semantic segmentation module performs pixel-level crack recognition and generates a segmented crack semantic pixel map.

[0055] The multi-objective genetic algorithm optimization module automatically optimizes the parameters of the threshold segmentation algorithm through the multi-objective genetic algorithm to improve the quality of pseudo-labeling.

[0056] The pixel-by-pixel fusion module is responsible for fusing the crack splitting and positioning results with the semantic segmentation results and outputting the final pseudo-annotated image.

[0057] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A crack detection and pseudo-labeling method based on deep learning and genetic algorithm, characterized by: include, Collect and prepare unlabeled raw image data as input to the processing pipeline; The original image is input into the crack detection and positioning module, the deep learning classification model is used for feature learning, and the Grad-CAM algorithm is used to output the location map of the cracks in the image; The original image is simultaneously input into the semantic segmentation module, and the deep learning model is used to identify cracks at the pixel level. In the semantic segmentation module, a multi-objective genetic algorithm is introduced to optimize the segmentation parameters. The output results of the crack detection and positioning module and the semantic segmentation module are fused, and the morphological closing operation is applied to the fused result image to fill the crack holes generated by the threshold segmentation; After morphological closing operation, the final crack pseudo-annotation result image is output.

2. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 1, characterized in that: The raw image data includes, using a high-resolution camera, collecting images covering various cracks, wear and tear, erosion structural problems, and images under different environmental conditions; at the same time, conducting on-site surveys of hydraulic structures to determine key areas and locations for image collection, and recording image metadata during the collection process; Key areas and locations include load-bearing support points, beam-column connections, and foundations; they also include sluice gates, valves, spillways with service functions, and areas where historical data shows frequent cracks or damage, and areas that have been repaired or reinforced; The quality of the collected images is evaluated, blurred, overexposed and underexposed images are removed, and the screened images are preprocessed, including denoising, enhancement and correction.

3. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 2, characterized in that: The output image includes a crack location map, which uses Grad-CAM to output a crack location heat map during crack detection and splitting to improve crack location accuracy; Pass the image forward through the CNN model to get the predicted score of the category: , in, represents the prediction score of the category, represents the weight of the connection layer, represents the input image, represents the CNN model, Represents the bias of the connection layer; Select the class with the highest prediction score and calculate the gradient of the prediction score with respect to the output of the last convolutional layer: , , in, Indicates the selection category. Indicates category The score, Represents the feature map of the convolutional layer; Perform global average pooling on the gradient values ​​to obtain weights, multiply the weights with the feature map and sum them to obtain the final class activation map: , , in, represents the weight, Representation feature map The dimension of represents the class activation map; The class activation map is upsampled to the size of the input image, generating a heatmap that can be overlaid on the original image.

4. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 3, characterized in that: The crack identification includes inputting the unlabeled original image into a semantic segmentation module, performing pixel-level crack identification, and generating a segmented crack semantic pixel map; Process the grayscale value of the original image and introduce a normalization function to ensure numerical stability: , , in, Represents the pixel point in the original image ( x , y ), represents the mean value of the image grayscale value, represents the normalization function, Indicates the minimum and maximum values ​​in the image; Using the deep learning segmentation network FCN, we can get the segmented crack semantic pixel map: , in, represents the activation function, Represents a segmentation model.

5. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 4, characterized in that: The optimization of the segmentation parameters includes automatically optimizing the parameters of the threshold segmentation algorithm through a multi-objective genetic algorithm to improve the quality of pseudo-labeling, and defining two optimization goals, the accuracy rate A and the coverage rate C' of the pseudo-labeling. , , , in, represents the optimization objective function, represents the weight factor, represents the threshold segmentation parameter, represents the genetic algorithm control parameter, represents the true label of the i-th data point, represents the label predicted by the threshold segmentation algorithm, represents the total number of data points, represents the indicator function, Represents a special label used to indicate a data point that the algorithm cannot determine.

6. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 5, characterized in that: The image fusion includes fusing the two binary images using a logical operation, and using a logical OR operation to ensure that all parts of the crack are marked; The fusion operation is represented by the following numerical expression: , in, is the fused image, It is the crack location map. is the threshold segmentation image, Represents a logical OR operation.

7. The crack detection and pseudo-labeling method based on deep learning and genetic algorithm according to claim 6, characterized in that: The morphological closing operation includes dilation and erosion operations, wherein the dilation operation dilates or grows the bright area in the image, and the erosion operation shrinks the bright area; The closing operation first performs an expansion operation and then an erosion operation. The specific expression is as follows: Assume A is the original image, B is the structural element, and the expansion operation and erosion operations They are defined as: , , in, represents the position of the structural element B on image A, represents the displacement of B; Morphological closing operation A ⋅ B Defined as: , After morphological closing operation, the final crack pseudo-annotation result image is output, and the similarity between the image and the real annotation is increased.

8. A system using the crack detection and pseudo-labeling method based on deep learning and genetic algorithm as claimed in any one of claims 1 to 7, characterized in that: It includes crack detection and positioning module, semantic segmentation module, multi-objective genetic algorithm optimization module, and pixel-by-pixel fusion module; The crack detection and positioning module uses a deep learning classification model to identify and locate cracks in the image; The semantic segmentation module performs pixel-level crack recognition and generates a segmented crack semantic pixel map; The multi-objective genetic algorithm optimization module automatically optimizes the parameters of the threshold segmentation algorithm through the multi-objective genetic algorithm to improve the quality of pseudo-labeling; The pixel-by-pixel fusion module is responsible for fusing the crack splitting and positioning results with the semantic segmentation results, and outputting the final pseudo-annotated image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Mask R-CNN-based reinforced concrete crack identification and measurement method and storage medium

    CN110738642A

  • Pavement crack detection method based on deep learning and web end positioning

    CN113628164A

  • Eye iris pigment spot texture automatic segmentation method

    CN115984550A

  • Image crack segmentation detection method and system based on AAMC-Net

    CN116342584A

  • Pavement crack detection method based on deep learning

    CN116503336A

Cited By

  • Fracture monitoring method and system based on image fusion and brightness enhancement and medium

    CN120219190A

  • Fissure Monitoring Method, System and Medium Based on Image Fusion and Brightness Enhancement

    CN120219190B