Pavement disease image automatic marking method, system and device and storage medium
Through the alternating collaborative training of disease image generation model and detection model, the full utilization of labelless data is achieved, the problem of insufficient utilization of labelless data in the existing technology is solved, and the training effect and accuracy of disease detection model is improved.
Patent Information
- Application Number
- CN202411156951.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-12-06
AI Technical Summary
The existing automatic labeling technology cannot make full use of labelless data, resulting in limited labeled data training effects and accuracy of detection model.
The alternating collaborative training method based on the disease image generation model and the disease detection model is adopted to update the labeled data set through sample expansion to realize the utilization of labelless data in the semi-supervised mode, and improve the training effect and accuracy of the detection model.
It effectively improves the training effect and detection accuracy of the disease detection model, makes full use of labelless data, and improves the labeling accuracy of the model.
Smart Images

Figure CN119091249A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, device and storage medium for automatic annotation of road surface disease images. Background Art
[0002] Under the influence of weather and external forces for a long time, the road surface condition will suffer different degrees of damage, resulting in various road surface diseases and potential traffic safety hazards. Timely discovery of road disease problems and repairs are of great significance in preventing further deterioration of road conditions and preventing various traffic safety problems.
[0003] With the development of deep learning in the field of computer vision, the proposed automatic pavement disease labeling technology, which collects pavement disease data of monitored sections, trains disease detection models, and then uses them for subsequent pavement disease detection, has replaced the traditional manual disease detection mode, which can save detection costs and improve detection efficiency to a certain extent. However, the application of existing automatic labeling technology must use a large amount of labeled data to train the detection model. Unlabeled data can only be used as the object to be detected and does not provide value in model training. In actual application scenarios, it is more difficult to obtain labeled data than unlabeled data. The training effect and labeling accuracy of the detection model will be lacking due to the small amount of labeled data involved in the training. Therefore, it is urgent to provide a method that can effectively use unlabeled data to train models, so as to better improve the training effect and detection accuracy of the detection model. Summary of the invention
[0004] The purpose of the present invention is to provide a method for automatic labeling of pavement disease images. By using an alternating collaborative training method to expand the samples of a labeled data set based on a disease image generation model and a disease detection model, it is possible to fully utilize unlabeled data in a semi-supervised mode and effectively improve the training effect and detection accuracy of the disease detection model.
[0005] In order to achieve the above objectives, it is necessary to provide a method, system, device and storage medium for automatic annotation of pavement disease images in response to the above technical problems.
[0006] In a first aspect, an embodiment of the present invention provides a method for automatically labeling a pavement defect image, the method comprising the following steps:
[0007] Constructing a pavement image dataset containing multiple disease categories; the pavement image dataset includes a labeled pavement image set and an unlabeled pavement image set;
[0008] Building a disease image generation model based on the labeled road image set, so as to perform a first sample expansion and update on the labeled road image set based on the disease image generation model;
[0009] Training and constructing a defect detection model according to the expanded and updated labeled road image set, so as to perform a second sample expansion and update on the labeled road image set according to the defect detection model and the unlabeled road image set;
[0010] According to the expanded and updated annotated road image set, the disease image generation model and the disease detection model are trained in a cyclic alternating iterative manner to obtain the disease detection model that meets the preset cyclic training termination condition as a road disease annotation model;
[0011] The pavement disease labeling model is used to label the pavement disease image to be labeled with disease identification and labeling to obtain a corresponding disease labeling image.
[0012] Furthermore, the step of constructing a pavement image dataset containing multiple disease categories includes:
[0013] Acquire an original pavement image dataset containing multiple types of damage; the original pavement image dataset also includes a labeled pavement damage dataset and an unlabeled pavement damage dataset;
[0014] Data enhancement processing is performed on the labeled pavement disease dataset and the unlabeled pavement disease dataset respectively to obtain a corresponding labeled pavement image set and an unlabeled pavement image set.
[0015] Furthermore, the step of training and constructing a disease image generation model based on the annotated road surface image set includes:
[0016] Cutting the damaged area of each labeled road surface image in the labeled road surface image set to obtain a labeled damaged image set;
[0017] According to the set of labeled disease images, a preset generative adversarial network is trained to obtain a disease image generation model.
[0018] Furthermore, the preset generative adversarial network is a WGAN-GP network.
[0019] Furthermore, the step of performing a first sample expansion and update on the annotated road surface image set according to the disease image generation model includes:
[0020] Generate a pseudo disease image set according to the labeled disease image set and the disease image generation model;
[0021] According to the pseudo defect image set, randomly replacing the defect area in each labeled road surface image in the labeled road surface image set to obtain a generated road surface image set;
[0022] The generated road surface image set is merged into the labeled road surface image set to perform a first sample expansion and update.
[0023] Furthermore, the step of randomly replacing the defective areas in each annotated road surface image in the annotated road surface image set according to the pseudo defective image set to generate the road surface image set includes:
[0024] Randomly sampling the pseudo disease image set to obtain a plurality of pseudo disease images to be embedded;
[0025] Through the Poisson fusion method, each pseudo-damage image to be embedded is covered to the damage area of the target road surface image to generate a corresponding generated road surface image; the target road surface image is a labeled road surface image randomly selected from the labeled road surface image set.
[0026] Furthermore, the step of performing a second sample expansion and updating on the labeled road image set according to the disease detection model and the unlabeled road image set includes:
[0027] Inputting the unlabeled pavement image set into the pavement disease labeling model to perform disease recognition detection to obtain a corresponding pavement disease detection result set;
[0028] A pavement image with pavement disease annotations in the pavement disease detection result set is obtained, and the pavement image is incorporated into the annotated pavement image set for a second sample expansion and update.
[0029] In a second aspect, an embodiment of the present invention provides a system for automatically annotating road surface disease images, the system comprising:
[0030] A data construction module is used to construct a pavement image data set containing multiple disease categories; the pavement image data set includes a labeled pavement image set and an unlabeled pavement image set;
[0031] A first expansion module is used to train and construct a disease image generation model based on the annotated road image set, so as to perform a first sample expansion and update on the annotated road image set based on the disease image generation model;
[0032] A second expansion module is used to train and construct a disease detection model according to the expanded and updated annotated road image set, so as to perform a second sample expansion and update on the annotated road image set according to the disease detection model and the unannotated road image set;
[0033] A loop iteration module, used for loop alternating and iteratively training the damage image generation model and the damage detection model according to the expanded and updated annotated road image set, so as to obtain the damage detection model that meets the preset loop training termination condition as the road damage annotation model;
[0034] The disease annotation module is used to perform disease identification and annotation on the road disease image to be annotated according to the road disease annotation model to obtain a corresponding disease annotation image.
[0035] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0036] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0037] The above-mentioned application provides a method, system, device and storage medium for automatic labeling of pavement disease images. The method realizes the construction of a labeled pavement image set and an unlabeled pavement image set containing multiple disease categories, constructs a disease image generation model based on the training of the labeled pavement image set to perform a first sample expansion and update of the labeled pavement image set according to the disease image generation model, constructs a disease detection model based on the training of the expanded and updated labeled pavement image set to perform a second sample expansion and update of the labeled pavement image set according to the pavement disease labeling model and the unlabeled pavement image set, and performs cyclic alternating iterative training on the disease image generation model and the disease detection model based on the expanded and updated labeled pavement image set to obtain a disease detection model that meets the preset cyclic training termination conditions as a pavement disease labeling model, and a technical solution for performing disease recognition and labeling on the pavement disease images to be labeled according to the pavement disease labeling model to obtain corresponding disease labeled images. Compared with the existing technology, this method for automatic labeling of pavement disease images can fully utilize unlabeled data in a semi-supervised mode through an alternating collaborative training method based on a disease image generation model and a disease detection model to expand the samples of the labeled data set, thereby effectively improving the training effect and detection accuracy of the disease detection model. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a schematic flow chart of a method for automatically labeling pavement damage images in an embodiment of the present invention;
[0039] Figure 2 is a schematic diagram of a process for obtaining a set of labeled disease images in an embodiment of the present invention;
[0040] Figure 3 is a schematic diagram of the structure of a WGAN-GP network in an embodiment of the present invention;
[0041] Figure 4 is a schematic diagram of the training process of the disease image generation model in an embodiment of the present invention;
[0042] Figure 5 is a schematic diagram of a process of performing the first sample expansion and update on a set of labeled disease images in an embodiment of the present invention;
[0043] Figure 6 is a schematic diagram of a process of performing a second sample expansion and update on a set of labeled disease images in an embodiment of the present invention;
[0044] Figure 7 is a schematic diagram of the structure of the automatic annotation system for pavement disease images according to an embodiment of the present invention;
[0045] Figure 8 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] The automatic pavement disease image labeling method provided by the present invention can be understood as a new automatic pavement disease labeling method applicable to a semi-supervised mode, based on the fact that the existing automatic pavement disease labeling technology cannot fully utilize a large amount of unlabeled data and the model labeling accuracy is limited by the application status of limited labeled data. The following embodiment will explain the automatic pavement disease image labeling method of the present invention in detail.
[0048] In one embodiment, Figure 1 As shown, a method for automatically labeling pavement disease images is provided, comprising the following steps:
[0049] S11. Construct a road image dataset containing multiple types of road damage; the road image dataset can be understood as a dataset collected based on field photography of roads with road damage such as cracks, potholes, rutting, looseness and subsidence by patrol vehicles. Existing datasets containing images of various road damages can also be used directly, and no specific limitation is made here. In order to meet the semi-supervised training requirements of the automatic annotation model, the road image dataset includes annotated road image sets and unannotated road image sets.
[0050] In order to increase the number of pavement image samples as much as possible and provide rich data support for the subsequent training and construction of the disease image generation model and the disease detection model, in this embodiment, the original pavement image dataset is preferably enhanced before use. Specifically, the steps of constructing a pavement image dataset containing multiple disease categories include:
[0051] An original pavement image dataset containing multiple disease categories is obtained; the original pavement image dataset includes both a labeled pavement disease dataset and an unlabeled pavement disease dataset; wherein the labeled pavement disease dataset can be obtained through manual labeling or other reliable labeling methods, and in order to save labeling costs, only a small part of the images in the original pavement image dataset can be labeled, and the remaining majority of the images are not labeled to obtain the unlabeled pavement disease dataset.
[0052] Data enhancement processing is performed on the labeled pavement disease dataset and the unlabeled pavement disease dataset respectively to obtain the corresponding labeled pavement image set X l and unlabeled road image set X u ; The specific data enhancement processing can be understood as performing data enhancement on each image sample in the original road image dataset through methods such as horizontal flipping, random cropping, scaling, rotation, and color space conversion to increase the number of samples and thus improve the robustness of the training model.
[0053] S12, constructing a damage image generation model based on the labeled road image set training, so as to perform a first sample expansion and update on the labeled road image set based on the damage image generation model; wherein the damage image generation model can be understood as a generation model that can be used to generate damage images and is obtained through training based on the labeled road image set. Specifically, the step of constructing a damage image generation model based on the labeled road image set training includes:
[0054] The diseased area of each labeled road image in the labeled road image set is cut out to obtain a labeled diseased image set; wherein the labeled diseased image set can be understood as a set of each labeled road image that has been cut out after the diseased area is cut out. Figure 2 The diseased area image dataset shown is obtained by cropping and resizing the diseased area.
[0055] According to the labeled defect image set, the preset generative adversarial network is trained to obtain a defect image generation model; wherein the preset generative adversarial network can in principle adopt any existing generative adversarial network. In order to ensure the reliability of the pavement defect images generated based on the defect image generation model as sample expansion data, this embodiment preferably adopts Figure 3The WGAN-GP network shown in the figure is used as the basic network architecture for training the disease image generation model. It introduces the Wasserstein distance to replace the JS (Jensen-Shannon) divergence in the traditional GAN to measure the difference between the generated samples and the real samples, which can effectively avoid the divergence problems such as gradient vanishing and gradient exploding. In addition, by introducing an additional gradient penalty mechanism in the loss term, the gradient of the discriminator is controlled within a reasonable range, which can effectively enhance the stability of the model training.
[0056] The WGAN-GP network in this embodiment includes Figure 3 The generator and discriminator shown, wherein the generator (G) includes a random noise addition module (batch size batch-size, noise vector dimension latent dim), a fully connected FC (input noise vector dimension latent dim, 128), a fully connected FC (128, 256), BatchNorm regularization, ReLU activation function, a fully connected FC (256, 512), BatchNorm regularization, ReLU activation function, a fully connected FC (512, 1024), BatchNorm regularization, ReLU activation function, a fully connected FC (input noise vector dimension latent dim) connected in sequence. dim, C*H*W), Tanh activation function and synthetic image output layer; the discriminator (D) includes a receiving module (receiving actual images and generated images), a flatten layer (batch size, C*H*W), a fully connected FC (C*H*W, 512), a ReLU activation function, a fully connected FC (512, 256), a ReLU activation function, a fully connected FC (256, 1) and a true discriminant score output layer. The corresponding process of training the preset generative adversarial network based on the labeled disease image set is as follows Figure 4 As shown, the loss function used for training the corresponding discriminator D is:
[0057]
[0058] Among them, x represents the real disease image sample; represents the pseudo diseased image samples generated by the generator, G represents the generator; P g and P r Represent the distribution of generated samples and real samples respectively; and They represent the gradient sampling samples and their distribution space obtained by randomly weighted averaging of generated samples and real samples respectively; λ represents the weighting coefficient of the gradient penalty term, which is generally 10; and Respectively represent the expectations of the discriminator for the generated samples, real samples and gradient penalty terms; represents the discriminator gradient of the gradient sampling sample; D represents the discriminator; L D represents the discriminant loss. The sum of the first two losses enables the discriminator to learn to distinguish between real diseases and pseudo diseases. The third loss term is the gradient penalty term of WGAN-GP.
[0059] The loss function used in the training of the generator G in this embodiment is:
[0060]
[0061] in, represents the pseudo diseased image sample generated by the generator G, P g Represents the generated sample distribution; Represents the discriminator probability expectation of the generated sample; the generator optimizes to reduce the discriminant loss of generated pseudo samples, thereby prompting the model to generate pseudo samples that are indistinguishable from the real ones.
[0062] The disease image generation model constructed by the above method can be directly used to expand the disease data. Figure 5 As shown, the step of performing a first sample expansion and update on the annotated road surface image set according to the disease image generation model includes:
[0063] A pseudo disease image set is generated according to the labeled disease image set and the disease image generation model; wherein the pseudo disease image set can be understood as a set of pseudo disease images generated by the disease image generation model by adding random noise to the labeled disease images in the labeled disease image set.
[0064] According to the pseudo diseased image set, the diseased areas in each labeled road surface image in the labeled road surface image set are randomly replaced to obtain a generated road surface image set; wherein the generated road surface image set can be understood as using some pseudo diseased images in the pseudo diseased image set to replace the diseased areas of the labeled road surface images in the labeled road surface image set, thereby obtaining a pseudo road surface image data set; specifically, according to the pseudo diseased image set, the diseased areas in each labeled road surface image in the labeled road surface image set are randomly replaced to obtain a generated road surface image set, including:
[0065] Randomly sampling the pseudo disease image set to obtain a plurality of pseudo disease images to be embedded;
[0066] Through the Poisson fusion method, each pseudo-damage image to be embedded is covered to the damage area of the target road surface image to generate a corresponding generated road surface image; the target road surface image is a labeled road surface image randomly selected from the labeled road surface image set.
[0067] In practical applications, such as Figure 5 As shown, the acquisition process of each generated road surface image in the above embodiment can be understood as randomly sampling each pseudo diseased image to be embedded from the pseudo diseased image set, randomly extracting the original image to be pasted back from the annotated road surface image set, and scaling the pseudo diseased image to be embedded to a suitable size according to the annotation of the original image, and then covering the diseased area in the original image through the Poisson fusion embedding method to generate a new diseased road surface image.
[0068] The generated road image set is merged into the annotated road image set for the first sample expansion update; that is, the road image set X is generated. g and a set of labeled road images X l Merge to obtain an expanded and updated set of annotated road images Used for subsequent disease detection model training.
[0069] S13, training and constructing a disease detection model based on the expanded and updated annotated road image set, so as to perform a second sample expansion and update on the annotated road image set based on the disease detection model and the unannotated road image set. The disease detection model can be understood as a model that can be used to locate and classify various types of diseases in road images. In this embodiment, the YOLOv5 network model is preferably used, and the corresponding loss function may include disease classification loss and positioning loss. The specific disease classification loss and positioning loss can be designed with reference to the prior art and will not be described in detail here.
[0070] In this embodiment, the second sample expansion and updating of the labeled road image set can be understood as further expanding the labeled road images used for the next round of iterative training of the disease image generation model; specifically, the step of performing the second sample expansion and updating of the labeled road image set according to the disease detection model and the unlabeled road image set includes:
[0071] Inputting the unlabeled pavement image set into the pavement disease labeling model to perform disease recognition detection to obtain a corresponding pavement disease detection result set;
[0072] Acquire a road surface image with road surface disease annotations in the road surface disease detection result set, and incorporate the road surface image into the annotated road surface image set for a second sample expansion and update;
[0073] This embodiment uses a defect detection model to detect unlabeled road surface images to mine more road surface defect image samples to expand the training data set of the defect image generation model, thereby further improving the reliability of the defect image generation model in generating defect images.
[0074] S14. Perform cyclic alternating iterative training on the disease image generation model and the disease detection model according to the expanded and updated annotated road image set, so as to obtain the disease detection model that meets the preset cyclic training termination condition as the road disease annotation model; wherein, the preset cyclic training termination condition can be set according to the actual application requirements, such as the detection accuracy of the disease detection model meets the expectations or reaches the target number of cyclic training times, etc., which is not specifically limited here. Correspondingly, the process of performing cyclic alternating iterative training on the disease image generation model and the disease detection model according to the expanded and updated annotated road image set can be understood as repeatedly executing the above-mentioned two sample expansion steps of first updating and training the disease image generation model using the expanded and updated annotated road image set, and generating pseudo disease images according to the updated disease image generation model, so as to expand the training set of the disease detection model again, and using the expanded annotated road image set to train and update the disease detection model, for disease classification and annotation of unlabeled road images.
[0075] S15. Perform disease identification and annotation on the road surface disease image to be annotated according to the pavement disease annotation model to obtain a corresponding disease annotation image; wherein the disease annotation image can be understood as a processing result obtained by directly using the pavement disease annotation model to perform disease classification and regional positioning on the road surface disease image to be annotated.
[0076] The embodiment of the present application constructs a labeled road image set and an unlabeled road image set containing multiple disease categories, trains and constructs a disease image generation model based on the labeled road image set to perform a first sample expansion and update of the labeled road image set based on the disease image generation model, trains and constructs a disease detection model based on the expanded and updated labeled road image set to perform a second sample expansion and update of the labeled road image set based on the road disease annotation model and the unlabeled road image set, and performs cyclic alternating iterative training on the disease image generation model and the disease detection model based on the expanded and updated labeled road image set to obtain a predicted disease image. A defect detection model with a cyclic training termination condition is set as a pavement defect labeling model, and a solution is proposed to perform disease identification and labeling on the pavement defect images to be labeled according to the pavement disease labeling model to obtain the corresponding disease labeled images. This effectively solves the application defects of existing automatic disease labeling technology that cannot make full use of a large amount of unlabeled data and the model labeling accuracy is limited by limited labeled data. Through the alternating collaborative training method of expanding the sample of the labeled data set based on the disease image generation model and the disease detection model, the full utilization of unlabeled data in the semi-supervised mode can be achieved, which effectively improves the training effect and detection accuracy of the disease detection model.
[0077] In order to verify the application effect of the method for automatic labeling of pavement disease images provided by the present invention, this embodiment takes the acquired pothole disease data as an example, and compares and verifies the automatic disease labeling method provided by the present invention with the existing automatic disease labeling. There are 2,000 labeled pavement images in the pothole disease data set, that is, as a labeled pavement image set, this part of the data can be divided into a test set and a training set at a ratio of 1:4, and the test set is used for subsequent verification of the automatic labeling effect. In addition, in order to meet the needs of semi-supervised scene training, the pothole disease data set also includes 5,000 pavement pictures that contain pothole diseases but do not contain any labeling information, which are used as an unlabeled pavement image set. As shown in Table 1, the pavement disease image labeling accuracy corresponding to the automatic labeling method for pavement disease images provided by the present invention is significantly higher than that of other existing automatic labeling methods:
[0078] Table 1 Comparison of automatic disease marking effects
[0079]
[0080] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.
[0081] In one embodiment, Figure 7 As shown, a road surface disease image automatic annotation system is provided, the system comprising:
[0082] Data construction module 1, used to construct a pavement image dataset containing multiple disease categories; the pavement image dataset includes a labeled pavement image set and an unlabeled pavement image set;
[0083] A first expansion module 2, used for training and constructing a disease image generation model according to the annotated road image set, so as to perform a first sample expansion and update on the annotated road image set according to the disease image generation model;
[0084] A second expansion module 3 is used to train and construct a disease detection model according to the expanded and updated annotated road image set, so as to perform a second sample expansion and update on the annotated road image set according to the disease detection model and the unannotated road image set;
[0085] A loop iteration module 4 is used to perform loop alternating iterative training on the disease image generation model and the disease detection model according to the expanded and updated annotated road image set, so as to obtain the disease detection model that meets the preset loop training termination condition as the road disease annotation model;
[0086] The disease annotation module 5 is used to perform disease identification and annotation on the road disease image to be annotated according to the road disease annotation model to obtain a corresponding disease annotation image.
[0087] For the specific limitations of the automatic pavement disease image annotation system, please refer to the limitations of the automatic pavement disease image annotation method mentioned above, and the corresponding technical effects can also be obtained equivalently, which will not be repeated here. Each module in the above-mentioned automatic pavement disease image annotation system can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.
[0088] Figure 8 FIG. 1 shows an internal structure diagram of a computer device in an embodiment, and the computer device may specifically be a terminal or a server. Figure 8 As shown, the computer device includes a processor, a memory, a network interface, a display, a camera and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for automatically marking pavement disease images is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0089] It can be understood by those skilled in the art that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have the same component arrangement.
[0090] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.
[0091] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0092] In summary, the embodiments of the present invention provide a method, system, device and storage medium for automatic labeling of pavement disease images, wherein the method for automatic labeling of pavement disease images realizes the construction of a labeled pavement image set and an unlabeled pavement image set containing multiple disease categories, constructs a disease image generation model based on the labeled pavement image set to perform a first sample expansion and update of the labeled pavement image set based on the disease image generation model, constructs a disease detection model based on the expanded and updated labeled pavement image set to perform a second sample expansion and update of the labeled pavement image set based on the pavement disease labeling model and the unlabeled pavement image set, and performs a second sample expansion and update of the labeled pavement image set based on the disease image generation model. The expanded and updated labeled pavement image set is used to perform cyclic alternating iterative training on the disease image generation model and the disease detection model to obtain a disease detection model that meets the preset cyclic training termination conditions as a pavement disease annotation model, and a technical solution for performing disease identification and annotation on the labeled pavement disease images according to the pavement disease annotation model to obtain corresponding disease annotated images. This method can fully utilize the unlabeled data in the semi-supervised mode through alternating collaborative training based on the disease image generation model and the disease detection model to expand the samples of the labeled data set, thereby effectively improving the training effect and detection accuracy of the disease detection model.
[0093] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0094] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.
Claims
1. A method for automatic annotation of pavement damage images. It is characterized in that The method comprises the following steps: Constructing a pavement image dataset containing multiple disease categories; the pavement image dataset includes a labeled pavement image set and an unlabeled pavement image set; Building a disease image generation model based on the labeled road image set, so as to perform a first sample expansion and update on the labeled road image set based on the disease image generation model; Training and constructing a defect detection model according to the expanded and updated labeled road image set, so as to perform a second sample expansion and update on the labeled road image set according to the defect detection model and the unlabeled road image set; According to the expanded and updated annotated road image set, the disease image generation model and the disease detection model are trained in a cyclic alternating iterative manner to obtain the disease detection model that meets the preset cyclic training termination condition as a road disease annotation model; The pavement disease labeling model is used to label the pavement disease image to be labeled with disease identification and labeling to obtain a corresponding disease labeling image.
2. The method for automatically labeling pavement damage images according to claim 1, It is characterized in that The step of constructing a pavement image dataset containing multiple disease categories includes: Acquire an original pavement image dataset containing multiple types of damage; the original pavement image dataset also includes a labeled pavement damage dataset and an unlabeled pavement damage dataset; Data enhancement processing is performed on the labeled pavement disease dataset and the unlabeled pavement disease dataset respectively to obtain a corresponding labeled pavement image set and an unlabeled pavement image set.
3. The method for automatically labeling pavement damage images according to claim 1, It is characterized in that The step of training and constructing a disease image generation model based on the annotated road surface image set comprises: Cutting the damaged area of each labeled road surface image in the labeled road surface image set to obtain a labeled damaged image set; According to the set of labeled disease images, a preset generative adversarial network is trained to obtain a disease image generation model.
4. The method for automatically labeling pavement damage images according to claim 3, It is characterized in that The preset generative adversarial network is a WGAN-GP network.
5. The method for automatically labeling pavement damage images according to claim 3, It is characterized in that The step of performing a first sample expansion and update on the annotated road surface image set according to the disease image generation model includes: Generate a pseudo disease image set according to the labeled disease image set and the disease image generation model; According to the pseudo defect image set, randomly replacing the defect area in each labeled road surface image in the labeled road surface image set to obtain a generated road surface image set; The generated road surface image set is merged into the labeled road surface image set to perform a first sample expansion and update.
6. The method for automatically labeling pavement damage images according to claim 5, It is characterized in that The step of randomly replacing the defective areas in each annotated road surface image in the annotated road surface image set according to the pseudo defective image set to generate the road surface image set comprises: Randomly sampling the pseudo disease image set to obtain a plurality of pseudo disease images to be embedded; Through the Poisson fusion method, each pseudo-damage image to be embedded is covered to the damage area of the target road surface image to generate a corresponding generated road surface image; the target road surface image is a labeled road surface image randomly selected from the labeled road surface image set.
7. The method for automatically labeling pavement damage images according to claim 1, It is characterized in that The step of performing a second sample expansion and updating on the labeled road image set according to the disease detection model and the unlabeled road image set comprises: Inputting the unlabeled pavement image set into the pavement disease labeling model to perform disease recognition detection to obtain a corresponding pavement disease detection result set; A pavement image with pavement disease annotations in the pavement disease detection result set is obtained, and the pavement image is incorporated into the annotated pavement image set for a second sample expansion and update.
8. An automatic annotation system for pavement disease images, It is characterized in that The system comprises: A data construction module is used to construct a pavement image data set containing multiple disease categories; the pavement image data set includes a labeled pavement image set and an unlabeled pavement image set; A first expansion module is used to train and construct a disease image generation model based on the annotated road image set, so as to perform a first sample expansion and update on the annotated road image set based on the disease image generation model; A second expansion module is used to train and construct a disease detection model according to the expanded and updated annotated road image set, so as to perform a second sample expansion and update on the annotated road image set according to the disease detection model and the unannotated road image set; A loop iteration module, used for loop alternating and iteratively training the damage image generation model and the damage detection model according to the expanded and updated annotated road image set, so as to obtain the damage detection model that meets the preset loop training termination condition as the road damage annotation model; The disease annotation module is used to perform disease identification and annotation on the road disease image to be annotated according to the road disease annotation model to obtain a corresponding disease annotation image.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Multi-threshold semi-supervised road abnormal disease intelligent detection method based on Yolo V5
CN114170480A
