Method and device for training crack image detection model based on active learning

By adopting active learning and difficult learning mechanisms in crack image detection and selecting high-value samples for annotation, the problem of high labeling cost in crack image detection is solved, and the model performance and efficiency are improved.

CN118918419AInactive Publication Date: 2024-11-08ZHEJIANG UNIV

Patent Information

Application Number
CN202411420766.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-12
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, deep learning models have high labeling costs and long training time in crack image detection tasks. They lack research methods and sample selection processes for active learning, making it difficult to effectively use unlabeled data to improve model performance.

Method used

The crack image detection model training method based on active learning is adopted. Through the difficult learning mechanism and the probability attention module, regions with uncertainty greater than the threshold are aggregated, the value of the image is calculated and sorted by value, key samples are selected for annotation, and the model is iteratively updated.

Benefits of technology

Significantly save labeling costs, improve model performance and generalization capabilities, optimize data set quality, reduce the impact of duplicate and low-quality samples, and improve training speed and detection efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918419A_ABST
    Figure CN118918419A_ABST
Patent Text Reader

Abstract

The invention discloses a training method and device of a crack image detection model based on active learning, the method selects a convolutional neural network as a preset detection model, the prediction difficulty of the convolutional neural network in crack detection is combined, and the most valuable crack image is automatically evaluated to request labeling, so that the detection accuracy is improved. Therefore, the model performance is improved to the greatest extent; the marking workload of the crack image is effectively reduced through an active learning method flow, and the crack detection efficiency and accuracy are improved; meanwhile, a probability attention module is utilized to enhance the attention of the convolutional neural network on a region which is relatively difficult to predict in the crack image; the convolutional neural network can better focus on pixels which fail in detection, and the accuracy of crack image evaluation is improved. According to the method, the problems of difficulty in labeling and insufficient high-quality data in a deep learning crack detection task are solved, and the labeling workload can be remarkably reduced and the detection performance can be improved by selecting the sample with the most information content for labeling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a training method and device for a crack image detection model based on active learning. Background Art

[0002] Deep learning methods have achieved remarkable results in image detection tasks, but the labor cost and training time required to annotate a large number of images have become challenges. In addition, with the increase in image acquisition devices, it has become more difficult to annotate all images. Simple, repetitive, and low-quality images have limited effect on improving network performance, but instead increase the time cost of training. Therefore, a method is needed to improve the quality of the dataset, reduce the annotation cost, and select high-value samples from a large amount of unlabeled data. Active learning is a supervised learning method that aims to maximize the performance of the model with minimal sample annotation. It uses a carefully designed sample selection strategy to select the most informative samples from unlabeled data, reduce the amount of data, computing power, and storage space, and optimize the performance of the model. However, in the field of crack images, there is currently a lack of research methods and clear sample selection processes for active learning. For different tasks, it is necessary to determine the appropriate sample selection strategy. When dealing with large-scale datasets, uncertainty-based sampling methods can be used to ensure classifier performance with minimal annotation costs. In the face of complex data sets, the use of representative strategies can promote sample diversity and reduce the selection of repeated samples. However, in the field of crack detection, in-depth research is still needed on how to use the feature recognition and prediction capabilities of convolutional neural networks to quantify the value and information content of unlabeled crack image samples and select image data that is most beneficial to network training. Summary of the invention

[0003] The purpose of the present invention is to provide a training method and device for a crack image detection model based on active learning to address the problems in the prior art of huge annotation costs and repetitive and cumbersome labeling work required to form a deep learning crack image dataset.

[0004] The objective of the present invention is achieved through the following technical solutions: A first aspect of an embodiment of the present invention provides a training method for a crack image detection model based on active learning, comprising the following steps: (1) Collect and preprocess images containing cracks on the surface of steel structures to construct a dataset; use some labeled crack images in the dataset to perform preliminary training on the preset detection model; (2) Use the preliminarily trained detection model to predict the unlabeled crack images in the dataset and output the predicted crack detection result feature map; (3) Based on the predicted crack detection result feature map, the probabilistic attention module of the difficulty learning mechanism is used to aggregate the areas where the uncertainty is greater than the preset uncertainty threshold to obtain a difficulty feature map; the uncertainty is quantitatively measured according to the probability value predicted by the detection model; (4) Using a sample selection strategy combined with a difficult feature map, the crack images are quantified and sorted according to their value from large to small; (5) Selecting some crack images with the highest ranking among the sorted crack images for annotation and obtaining their corresponding crack marks; (6) Using the crack images annotated in step (5) and their corresponding crack labels as training samples to train and update the preliminarily trained detection model, and using the currently trained detection model to update the unlabeled data set; (7) Repeat steps (2) to (6) to iteratively update the detection model until the preset training stop condition is reached.

[0005] Furthermore, the step (1) specifically includes: Collecting images containing cracks on the surface of the steel structure, and preprocessing the collected crack images so that the preprocessed crack images meet the image input requirements of the preset detection model; wherein the preprocessing includes size adjustment, normalization and smoothing; Constructing a data set based on the preprocessed crack images; wherein the data set includes cracks of different types and sizes and crack images under various background and lighting conditions; Manually annotate some crack images in the dataset to distinguish between background and crack pixels; Some labeled crack images are input into the preset detection model for preliminary training to complete the crack detection task and obtain a preliminarily trained detection model.

[0006] Furthermore, the step (2) specifically includes: The preliminarily trained detection model obtained in step (1) is loaded, and the unlabeled crack images in the data set are provided to the preliminarily trained detection model as input to obtain a corresponding crack detection result feature map; wherein the crack detection result feature map includes the location and probability information of the cracks in the crack image.

[0007] Furthermore, the difficulty learning mechanism includes a first branch and a second branch, wherein the first branch is a general semantic segmentation network, which is used to implement the crack detection task, and uses the predicted crack detection result feature map as the prediction result to compare with the labeled true value to generate an error mask; the second branch is a semantic difficulty branch, which is used to learn the difficult feature map of semantic segmentation using the error mask, and the second branch includes an improved probabilistic attention module and a convolutional layer, and the crack detection result feature map passes through the probabilistic attention module and the convolutional layer in sequence to obtain the difficult feature map.

[0008] Furthermore, the probabilistic attention module is used to aggregate similar pixels in the crack image and make them have similar difficulty scores; The crack detection result feature map is input into the probabilistic attention module, and first a spatial matrix operation is performed to obtain a probabilistic attention matrix. The crack detection result feature map is multiplied by the probabilistic attention matrix to obtain a first feature map. The first feature map is then fused and added with the crack detection result feature map to obtain a second feature map, so as to aggregate areas where the uncertainty is greater than the preset uncertainty threshold.

[0009] Furthermore, the step (4) specifically includes: The sample selection strategy calculates the difficulty and uncertainty of the crack image based on the error mask and the difficulty feature map, where the difficulty is calculated by the difficulty feature map and the uncertainty is calculated by the prediction probability of the detection model; the evaluation indicators of difficulty and uncertainty are weighted and fused to obtain a comprehensive evaluation indicator; the value of each crack image is quantified according to the comprehensive evaluation indicator, and a value score is assigned to each crack image through the entropy function to represent its value in active learning; according to the value score of the crack image, the crack images are sorted in descending order of the value score to obtain the sorted crack image.

[0010] Furthermore, the step (5) specifically includes: According to the total number of the sorted crack images, some of the crack images with the highest sorting are selected in proportion; the selected crack images are annotated using an image annotation tool to obtain corresponding crack marks.

[0011] Furthermore, the step (6) specifically includes: The crack image annotated in step (5) and its corresponding crack mark are used as training samples to construct a training set; the preliminarily trained detection model is trained using the annotated training samples in the training set, and gradient update is performed according to the total loss function of the detection model through the back propagation algorithm and the optimizer to adjust the parameters of the detection model and obtain the currently trained detection model; wherein the total loss function of the detection model includes semantic segmentation loss, semantic difficulty loss and detection loss, the semantic segmentation loss selects the Dice loss function, the semantic difficulty loss selects the inverse weighted binary cross entropy loss, and the detection loss selects the binary cross entropy loss; The currently trained detection model is used to predict the unlabeled crack images, and the prediction results are used as new crack labels. By setting a threshold, the samples corresponding to the prediction confidence greater than the threshold are added to the labeled dataset to complete the update of the unlabeled dataset.

[0012] A second aspect of an embodiment of the present invention provides a training device for a crack image detection model based on active learning, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned training method for a crack image detection model based on active learning.

[0013] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned training method for the crack image detection model based on active learning.

[0014] The beneficial effects of the present invention are as follows: the present invention can significantly save the labeling cost; active learning can select the most valuable samples from unlabeled data for labeling through a sample selection strategy, thereby reducing the workload and cost of labeling; active learning can select samples for labeling in a targeted manner, so that the model can learn key features and difficult examples more intensively, thereby improving the model performance and generalization ability; the present invention selects samples with the largest amount of information for labeling, and compared with randomly selecting samples for labeling, the same performance level can be achieved with a smaller number of labeled samples, thereby speeding up the training speed; through the sample selection strategy, representative and diverse samples can be screened out, the quality of the data set is optimized, the training data is made more comprehensive and balanced, and the influence of repeated and low-quality samples is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of the route of the training method of the crack image detection model based on active learning of the present invention; Figure 2 is a flowchart of the difficult learning mechanism used in the present invention; Figure 3 Schematic diagram of the probabilistic attention module in the present invention; Figure 4 It is a schematic diagram of the iterative update result of the model described in the present invention; Figure 5 It is a structural schematic diagram of a training device for a crack image detection model based on active learning of the present invention. DETAILED DESCRIPTION

[0016] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the above general description and the detailed description that follows are merely exemplary and explanatory and cannot limit the present application.

[0017] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that, although the term first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of..." or "when..." or "in response to determination". Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also include other elements not clearly listed, or also include elements inherent to such process and method. In the absence of more restrictions, the elements limited by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0019] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0020] See also Figure 1 The training method of the crack image detection model based on active learning of the present invention specifically comprises the following steps: (1) Collect and preprocess images containing surface cracks of steel structures to construct a dataset; use some labeled crack images in the dataset to perform preliminary training on the preset detection model.

[0021] Specifically, a large number of data sets containing images of cracks on the surface of steel structures can be obtained through on-site photography, public data sets or other available resources. To ensure the diversity of the data set, the data set includes cracks of different types and sizes and crack images under various background and lighting conditions. The collected crack images are preprocessed so that the preprocessed crack images meet the image input requirements of the preset detection model; the preprocessing includes size adjustment, normalization, smoothing, etc. Then, a data set is constructed based on the preprocessed crack images. Then, a small part of the crack images in the data set are manually annotated to distinguish between background and crack pixels. Then, a small number of annotated crack images are used to preliminarily train the preset detection model, and some of the annotated crack images are input into the preset detection model for training to complete the crack detection task and obtain a preliminarily trained detection model.

[0022] Furthermore, the detection model is specifically: based on an appropriate deep learning framework, such as TensorFlow, PyTorch, etc., it is generally preferred to use a convolutional neural network (CNN) built based on convolutional layers and pooling layers to create a detection model for detecting crack images; of course, other common neural networks can also be used to create a detection model for detecting crack images, such as a LeNet network, an AlexNet network, etc.

[0023] (2) Use the preliminarily trained detection model to predict the unlabeled crack images in the dataset and output the predicted crack detection result feature map.

[0024] Specifically, load the preliminarily trained detection model, use the deep learning framework to load the detection model for detecting crack images obtained in step (1), and ensure that the loaded detection model is consistent with the network structure and parameters used during training. Provide the unlabeled crack images in the dataset to the preliminarily trained detection model as its input, and use the preliminarily trained detection model to predict the unlabeled crack images. Through forward propagation, pass the unlabeled crack images into the detection model, and obtain the output results. Extract the crack detection result feature maps as the crack detection results. These feature maps can reflect the location and probability information of the cracks in the crack image.

[0025] It should be noted that when using the preliminarily trained detection model to predict the unlabeled crack images in the data set, the prediction results of the preliminarily trained detection model for the crack images need to be retained. The prediction results include the numerical value of the probability of whether each pixel in the crack image belongs to the crack category.

[0026] (3) Based on the predicted crack detection result feature map, the probabilistic attention module of the difficult learning mechanism is used to aggregate the areas where the uncertainty is greater than the preset uncertainty threshold to obtain the difficult feature map. The uncertainty is quantitatively measured based on the probability value predicted by the detection model. For example, when the predicted probability of a pixel is close to 0.5, the detection model has low confidence in the classification of the pixel, indicating that the uncertainty is high.

[0027] It should be noted that the difficult learning mechanism is proposed to solve the obvious semantic difficulty differences between different regions in the image caused by category imbalance and shape differences in the crack image detection task. In order to capture this difference, the error mask is first generated using the predicted crack detection result feature map. The error mask reflects the confusion between the background and the cracks faced by the detection model at the pixel level. These prediction errors are related to the detection difficulty of the crack image itself and the detection of the detection model. In order to quantitatively evaluate this detection difficulty, this embodiment constructs a multi-task dual-branch structure, namely the difficult learning mechanism described in the present invention.

[0028] Furthermore, the difficult learning mechanism includes two branches, namely: a first branch and a second branch, such as Figure 2 As shown in the figure, the first branch is a general semantic segmentation network, which is used to implement the crack detection task, and compares the predicted crack detection result feature map with the labeled true value as the prediction result to generate an error mask; the second branch is a semantic difficulty branch, which is used to learn the difficult feature map of semantic segmentation using the error mask. Among them, the second branch includes an improved probabilistic attention module and a convolutional layer. The crack detection result feature map passes through the probabilistic attention module and the convolutional layer in turn to obtain the difficult feature map.

[0029] Furthermore, the probabilistic attention module can aggregate similar pixels in the crack image and make them have similar difficulty scores, such as Figure 3 As shown, the predicted crack detection result feature map is input into the probabilistic attention module of the second branch, and a spatial matrix operation is first performed to obtain a probabilistic attention matrix; the predicted crack detection result feature map is multiplied by the probabilistic attention matrix to obtain a first feature map; the first feature map is then fused and added with the predicted crack detection result feature map to obtain a second feature map, so as to aggregate areas where uncertainty is greater than a preset uncertainty threshold.

[0030] In addition, in order to drive the training process of the detection model, it is necessary to use a loss function to include both branches in the parameter update. To this end, corresponding loss functions are designed for the two branches, namely, semantic segmentation loss and semantic difficulty loss. In the first branch, the Dice loss function is introduced for the crack detection task as a semantic segmentation loss. For the second branch, that is, for the processing of difficult feature maps, this embodiment proposes the inverse weighted binary cross entropy loss defined below, which is used as a semantic difficulty loss. Inverse weighting is used to balance the imbalance between the correct and incorrect areas in the error mask.

[0031] (4) Using a sample selection strategy combined with a difficulty feature map, the crack images are quantified in value and sorted according to their value from large to small.

[0032] It should be understood that the predicted crack detection result feature map is obtained using the preliminarily trained detection model, and then the predicted crack detection result feature map is used to obtain the difficult feature map through the difficult learning mechanism described in step (3). The difficult feature map can reflect the distribution of uncertainty and difficulty in the crack image.

[0033] Specifically, the sample selection strategy calculates the difficulty and uncertainty of the crack image according to the error mask and the difficulty feature map, wherein the difficulty is calculated by the difficulty feature map, and the uncertainty is calculated by the prediction probability of the detection model; the evaluation indicators of the difficulty and uncertainty are weighted and fused to obtain a comprehensive evaluation indicator; the value of each crack image is quantified according to the comprehensive evaluation indicator, and a value score is assigned to each crack image through the entropy function to represent its value in active learning; according to the value score of the crack image, the crack images are sorted in order from large to small to obtain the sorted crack images. Among them, the larger the value score of the crack image, the greater the uncertainty and value of the crack image, that is, the crack image has higher uncertainty and difficulty, and therefore is more valuable. Crack images with high uncertainty and difficulty are preferentially selected for the training and optimization of the detection model, which can ensure that crack images with higher value are first selected for annotation or detection model training in active learning, thereby improving the performance and efficiency of the detection model.

[0034] It should be noted that the difficulty and uncertainty information of the crack image are combined. Specifically, the uncertainty and difficulty evaluation indicators calculated can be weighted and fused through certain rules or entropy functions. In this way, the difficulty and uncertainty information in the crack image can be comprehensively considered, and then the value of each crack image can be quantified according to the comprehensive evaluation indicators. By quantifying the value of each crack image, a numerical value can be assigned to each crack image, and a numerical value reflecting the value of the crack image can be obtained, indicating its value in active learning. Afterwards, according to the value score of the crack image, the samples that are most helpful for the training of the detection model are preferentially selected.

[0035] (5) Select some crack images with the highest ranking among the sorted crack images for professional annotation to obtain their corresponding crack marks.

[0036] Specifically, according to the total number of sorted crack images, some of the crack images with the highest sorting are selected in proportion, for example, the top 10%, the top 20% or other proportion of crack images can be selected for professional annotation; the selected crack images are sent to professional annotators or teams to request corresponding crack labels; the professional annotators will carefully observe each crack image and annotate them according to the location and shape of the cracks. The annotation can be completed using specific annotation tools or software, such as the open source image annotation tool LabelImg, to ensure the accuracy and consistency of the crack labeling; after the professional annotators complete the annotation, they will return the crack labeling results to the active learning system or data management platform to associate each crack image with its corresponding crack label for subsequent detection model training and optimization.

[0037] It should be understood that, in the current step, when professionally labeling the crack images, some crack images with larger value scores are selected for labeling to obtain their crack marks.

[0038] (6) Using the crack images and their corresponding crack labels professionally annotated in step (5) as training samples, the initially trained detection model is trained and updated to improve its performance, and the currently trained detection model is used to update the unlabeled data set.

[0039] Specifically, the crack images professionally annotated in step (5) and their corresponding crack labels form a training set. In addition, unlabeled crack data sets can also be added to the training set to expand the scale of training data; the detection model preliminarily trained in step (1) is trained using the annotated training samples in the training set, and the parameters of the detection model are adjusted by gradient update based on the total loss function of the detection model through the back propagation algorithm and the optimizer to obtain the currently trained detection model, which is conducive to improving the detection ability of the detection model for cracks. Among them, the total loss function of the detection model includes semantic segmentation loss, semantic difficulty loss and detection loss. The semantic segmentation loss selects the Dice loss function, the semantic difficulty loss selects the inverse weighted binary cross entropy loss, and the detection loss selects the binary cross entropy loss. These are all commonly used loss functions and will not be repeated here. The currently trained detection model is used to predict the unlabeled crack images, and the prediction results are used as new crack labels. By setting a threshold, the samples corresponding to the prediction confidence greater than the threshold are added to the labeled data set to complete the update of the unlabeled data set.

[0040] (7) Repeat steps (2) to (6) to iteratively update the detection model until a preset training stop condition is reached. The preset training stop condition is reaching the labeling budget or the detection model performance convergence or reaching the maximum number of training rounds.

[0041] Specifically, before starting the iterative update, one or more stop conditions need to be set to determine when to end the iteration. For example, a labeling budget can be set, that is, the iteration is stopped after a certain number of labeled samples are reached. In addition, the convergence condition of the detection model performance can be set, and the iteration is stopped when the performance of the detection model is no longer significantly improved. By continuously repeating steps (2) to (6) until the preset stop condition is reached, multiple iterative updates can be performed to continuously improve the detection model and data set. Such an iterative process helps to gradually improve the performance of the crack detection model and make full use of the limited labeling budget. At the same time, according to the performance convergence of the detection model and the stop condition, the number and scale of iterative updates can be flexibly controlled to meet specific needs and restrictions. Figure 4 The figure shows how the performance (prediction accuracy) of the detection model changes over multiple iterations, from 10% of the labeled data to 40% of the annotation budget (stopping condition).

[0042] It should be pointed out that the examples use structural surface cracks for illustration, but the present invention is not limited to these, and may also include steel structures, composite structures and other buildings, concrete spalling, structural surface potholes and other structural damage.

[0043] In summary, the present invention proposes a high-quality crack image selection process based on active learning, which uses a difficult learning mechanism and an improved probabilistic attention module to accurately evaluate difficult-to-detect crack image samples, greatly reducing the labeling cost and ensuring crack detection performance. This method combines the prediction difficulty of convolutional neural networks in crack detection, and requests annotation by automatically evaluating the most valuable crack images, thereby maximizing the performance of the detection model; the active learning method process proposed in the present invention can effectively reduce the workload of labeling crack images and improve the efficiency and accuracy of crack detection; the method also improves the probabilistic attention module to enhance the network's attention to the more difficult-to-predict areas in the crack image, especially the slender features of the crack. By improving the probabilistic attention mechanism, the detection model can better focus on pixels that failed to detect, and improve the accuracy of crack image evaluation.

[0044] Corresponding to the above-mentioned embodiment of the training method of the crack image detection model based on active learning, the present invention also provides an embodiment of the training device of the crack image detection model based on active learning.

[0045] See also Figure 5 An embodiment of the present invention provides a training device for a crack image detection model based on active learning, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the training method for the crack image detection model based on active learning in the above embodiment.

[0046] The embodiment of the training device of the crack image detection model based on active learning of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 5 As shown, it is a hardware structure diagram of any device with data processing capability where the training device of the crack image detection model based on active learning of the present invention is located, except Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0047] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0048] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0049] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the training method of the crack image detection model based on active learning in the above embodiment is implemented.

[0050] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0051] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a crack image detection model based on active learning, characterized in that: The following steps are involved: (1) Collect and preprocess images containing cracks on the surface of steel structures to construct a dataset; use some labeled crack images in the dataset to perform preliminary training on the preset detection model; (2) Use the preliminarily trained detection model to predict the unlabeled crack images in the dataset and output the predicted crack detection result feature map; (3) Based on the predicted crack detection result feature map, the probabilistic attention module of the difficulty learning mechanism is used to aggregate the areas where the uncertainty is greater than the preset uncertainty threshold to obtain a difficulty feature map; the uncertainty is quantitatively measured according to the probability value predicted by the detection model; (4) Using a sample selection strategy combined with a difficult feature map, the crack images are quantified and sorted according to their value from large to small; (5) Selecting some crack images with the highest ranking among the sorted crack images for annotation and obtaining their corresponding crack marks; (6) Using the crack images annotated in step (5) and their corresponding crack labels as training samples to train and update the preliminarily trained detection model, and using the currently trained detection model to update the unlabeled data set; (7) Repeat steps (2) to (6) to iteratively update the detection model until the preset training stop condition is reached.

2. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The step (1) specifically includes: Collecting images containing cracks on the surface of the steel structure, and preprocessing the collected crack images so that the preprocessed crack images meet the image input requirements of the preset detection model; wherein the preprocessing includes size adjustment, normalization and smoothing; Constructing a data set based on the preprocessed crack images; wherein the data set includes cracks of different types and sizes and crack images under various background and lighting conditions; Manually annotate some crack images in the dataset to distinguish between background and crack pixels; Some labeled crack images are input into the preset detection model for preliminary training to complete the crack detection task and obtain a preliminarily trained detection model.

3. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The step (2) specifically includes: The preliminarily trained detection model obtained in step (1) is loaded, and the unlabeled crack images in the data set are provided to the preliminarily trained detection model as input to obtain a corresponding crack detection result feature map; wherein the crack detection result feature map includes the location and probability information of the cracks in the crack image.

4. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The difficulty learning mechanism includes a first branch and a second branch, wherein the first branch is a general semantic segmentation network, which is used to implement the crack detection task, and uses the predicted crack detection result feature map as the prediction result to compare with the labeled true value to generate an error mask; the second branch is a semantic difficulty branch, which is used to learn the difficult feature map of semantic segmentation using the error mask, and the second branch includes an improved probabilistic attention module and a convolutional layer. The crack detection result feature map passes through the probabilistic attention module and the convolutional layer in sequence to obtain the difficult feature map.

5. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The probabilistic attention module is used to aggregate similar pixels in the crack image and make them have similar difficulty scores; The crack detection result feature map is input into the probabilistic attention module, where the spatial matrix operation is first performed to obtain the probabilistic attention matrix; The crack detection result feature map is multiplied by the probability attention matrix to obtain a first feature map; the first feature map is then fused and added with the crack detection result feature map to obtain a second feature map to aggregate areas where the uncertainty is greater than a preset uncertainty threshold.

6. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The step (4) specifically includes: The sample selection strategy calculates the difficulty and uncertainty of the crack image based on the error mask and the difficulty feature map, where the difficulty is calculated by the difficulty feature map and the uncertainty is calculated by the prediction probability of the detection model; the evaluation indicators of difficulty and uncertainty are weighted and fused to obtain a comprehensive evaluation indicator; the value of each crack image is quantified according to the comprehensive evaluation indicator, and a value score is assigned to each crack image through the entropy function to represent its value in active learning; according to the value score of the crack image, the crack images are sorted in descending order of the value score to obtain the sorted crack image.

7. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The step (5) specifically includes: According to the total number of the sorted crack images, some of the crack images with the highest sorting are selected in proportion; the selected crack images are annotated using an image annotation tool to obtain corresponding crack marks.

8. The training method of the crack image detection model based on active learning according to claim 1 is characterized in that: The step (6) specifically includes: The crack image annotated in step (5) and its corresponding crack mark are used as training samples to construct a training set; the preliminarily trained detection model is trained using the annotated training samples in the training set, and gradient update is performed according to the total loss function of the detection model through the back propagation algorithm and the optimizer to adjust the parameters of the detection model and obtain the currently trained detection model; wherein the total loss function of the detection model includes semantic segmentation loss, semantic difficulty loss and detection loss, the semantic segmentation loss selects the Dice loss function, the semantic difficulty loss selects the inverse weighted binary cross entropy loss, and the detection loss selects the binary cross entropy loss; The currently trained detection model is used to predict the unlabeled crack images, and the prediction results are used as new crack labels. By setting a threshold, the samples corresponding to the prediction confidence greater than the threshold are added to the labeled dataset to complete the update of the unlabeled dataset.

9. A training device for a crack image detection model based on active learning, comprising one or more processors and a memory, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the training method of the crack image detection model based on active learning as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, it is used to implement the training method of the crack image detection model based on active learning described in any one of claims 1-8.

Citation Information

Patent Citations

  • Industrial product surface defect detection method and system based on deep active learning

    CN111833313A

  • Tower crane crack image detection method

    CN112241950A

  • Ballastless track slab crack measurement method based on multi-scale cooperative deep learning

    CN114494240A

  • Pavement crack detection method of improved FPHBN network structure

    CN114638802A

  • Image classification method and device, computer equipment and storage medium

    CN115775322A

Cited By

  • Water quality detection method and system based on combination of machine learning and spectrophotometry

    CN120028268A

  • A water quality detection method and system based on machine learning combined with spectrophotometry

    CN120028268B

  • Crack detection model training method and device based on dynamic multi-strategy active learning

    CN121121343A