Crack image generation method based on cyclic consistency constraint and diffusion model
By generating realistic crack images through cyclic consistency constraints and diffusion models, the problems of data scarcity and poor model adaptability in surface crack identification are solved, and efficient crack identification in complex geological scenarios is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-19
AI Technical Summary
In existing technologies, methods for expanding datasets for surface crack identification suffer from problems such as data scarcity, high annotation costs, and poor model robustness and adaptability. In particular, it is difficult to generate realistic and diverse images of natural bare ground cracks in complex geological scenarios, resulting in poor training performance of deep learning models.
A method based on cyclic consistency constraints and a diffusion model is adopted. The encoder maps the image to a low-dimensional latent space, constructs a bidirectional diffusion generation framework, and uses cyclic consistency constraints and adversarial loss function to train the generator to generate realistic crack images.
With small sample sizes and low costs, intelligent and controllable realistic crack datasets are generated, which improves the crack recognition performance of deep learning models in complex geological scenarios and solves the problems of data scarcity and poor model adaptability.
Smart Images
Figure CN122244212A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image synthesis technology, and more specifically, to a method for generating crack images based on cyclic consistency constraints and a diffusion model. Background Technology
[0002] The area, length, width, and propagation direction of surface cracks are key indicators for assessing geological safety. Accurate crack detection provides crucial information for geological disaster early warning. However, traditional surface crack detection methods largely rely on sensors or 3D laser scanners to monitor discrete point displacements, paying insufficient attention to the crack's morphological characteristics and thus failing to meet the demands of precise detection. With the development of UAV and computer vision technologies, using UAV imagery combined with deep learning models for surface crack identification has become a research hotspot due to its advantages of being non-contact, low-cost, easy to deploy, and having wide coverage, significantly improving the efficiency and accuracy of crack identification.
[0003] However, current research on deep learning-based surface crack recognition using UAV imagery is generally limited by difficulties in obtaining high-quality datasets, high annotation costs, and a scarcity of open-source data, directly restricting the training performance and generalization ability of deep learning models. Existing methods for dataset expansion in crack recognition mainly fall into three categories, all of which have significant technical shortcomings and cannot effectively solve the aforementioned problems: (1) Traditional enhancement methods based on geometric transformation and photometric transformation: These methods expand the data through geometric operations such as rotation, flipping, scaling and photometric operations such as brightness and contrast adjustment. Although they are simple to implement and have low computational cost, they can only perform simple transformations on existing samples and cannot generate new samples. They have limited effect on improving the robustness of the model and are also prone to model overfitting. (2) Synthetic data method based on generative model: With the help of deep learning techniques such as generative adversarial network and diffusion model, realistic crack images can be synthesized, which alleviates the problem of data scarcity to a certain extent. However, existing research focuses on the generation of cracks in artificial structures such as road surface and concrete. Research on the generation of realistic crack images in natural bare land such as soil and rock slope is very scarce, which is difficult to meet the model training needs in complex geological scenarios. (3) Domain-adaptive transfer learning method: The general features of the concrete crack dataset are transferred to the natural surface crack data by using the pre-trained model. This can make up for the lack of data in small samples. However, this method is highly dependent on the feature similarity between the source domain and the target domain. When facing areas with large differences in geological conditions and climate, the domain adaptability of the model is significantly insufficient.
[0004] Meanwhile, deep learning-based crack recognition mainly falls into two major technical routes: target detection and semantic segmentation. The former can obtain the bounding rectangle of the crack, while the latter can achieve pixel-level annotation of the crack. However, existing research still has significant shortcomings: on the one hand, most studies only focus on data cleaning and model optimization for specific regions, rarely paying attention to the robustness and adaptability of the model under different regions and geological conditions, resulting in poor scene transferability of the model; on the other hand, existing research ignores the inherent defects of crack data in natural scenes—cracks are discretely distributed, and the vast majority of areas in the dataset are crack-free backgrounds, forming a serious imbalance between positive and negative samples. How to use a small number of positive crack samples to train a high-precision crack recognition model has become a technical problem that urgently needs to be solved.
[0005] In summary, existing methods for expanding datasets for crack identification all have limitations, and the training and optimization of crack identification models have not solved the problems of poor scene adaptability and imbalance of positive and negative samples. There is an urgent need for an efficient crack image generation method to generate high-fidelity and diverse natural bare ground crack images at low cost, supplement high-quality training data, and improve the crack identification performance of deep learning models in complex geological scenes. Summary of the Invention
[0006] To address the aforementioned issues, this application provides a crack image generation method based on cyclic consistency constraints and a diffusion model, aiming to solve problems such as the difficulty in acquiring traditional bare land crack datasets, high aerial photography costs, and low fidelity of generative methods.
[0007] The first aspect of this invention provides a crack image generation method based on a cyclic consistency constraint and a diffusion model, comprising: Obtain a training image set and its corresponding mask, wherein the training image set includes cracked images and crack-free images; The images in the training image set are mapped to a low-dimensional latent space through an encoder to obtain a training latent representation, and the corresponding mask is downsampled to obtain a training latent mask; A bidirectional diffusion generation framework is constructed, the framework including a first generation process and a second generation process; the first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image based on the trained latent mask, and the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image based on the trained latent mask; In the low-dimensional latent space, the bidirectional diffusion generation framework is trained using a cycle consistency constraint; wherein, the cycle consistency constraint is used to ensure that the result obtained after the image undergoes successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. The target image and its corresponding mask are obtained. The target image is mapped to a low-dimensional latent space through an encoder to obtain the source latent representation. The corresponding mask is downsampled to obtain the latent mask. Based on the source latent representation and latent mask, iterative denoising is performed through the target generation process determined in the trained bidirectional diffusion generation framework to generate the target latent representation; The latent representation of the target is reconstructed into the image space using a decoder to obtain the fused crack image.
[0008] In one alternative implementation, the training of the bidirectional diffusion generation framework is constrained by a cycle consistency loss function, which calculates the difference between the reconstructed image obtained after the image has passed through the second generation process and the first generation process in sequence, and the initial latent representation of the image when it enters the cycle.
[0009] In one alternative implementation, a first discriminator is also used when training the bidirectional diffusion generation framework; the first discriminator discriminates the potential representation of the cracked image synthesized by the first generation process and calculates the adversarial loss using the least squares GAN loss function.
[0010] In one alternative implementation, the bidirectional diffusion generation framework is trained in two stages: In the first stage, the second generation process is pre-trained using a paired set of training images to optimize the diffusion model loss; In the second stage, the parameters of the second generation process are fixed, and the first generation process and the first discriminator are trained using an unpaired training image set; wherein, the optimized total loss function is:
[0011] in, For Stage 2, the generator-side total loss is... To predict the loss for diffused noise, For latent space cyclic consistency loss, For the generator's adversarial loss, These are the weight hyperparameters for the corresponding loss term.
[0012] In one optional implementation, the corresponding mask is downsampled using max pooling to obtain a potential mask that matches the size of the low-dimensional potential space.
[0013] in, For adaptive max pooling, downsampling is performed to the specified output size. For latent space masking, This is the mask after boundary expansion.
[0014] In one optional implementation, the process of performing iterative denoising includes: In each denoising step, the current noise latent representation, time step, source latent representation, and latent mask are input into the denoising network to obtain the predicted denoising result; The source latent representation is used to calculate the known latent representation corresponding to the current time step through a forward diffusion process; The predicted denoising result and the known latent representation are mixed element-wise according to the latent mask. The predicted denoising result is used to fill the inner region of the latent mask, and the known latent representation is used to fill the outer region of the latent mask to obtain the denoising result of the current step, which is used as the current noise latent representation for the next denoising step. Repeat the above steps until the preset number of denoising steps are completed to obtain the final latent representation of the target.
[0015] In one optional implementation, the weighted mixing operation is as follows:
[0016] in, For the updated current latent variable, "Assignment / in-place update symbol, For the model to predict latent variables in the current step, Use as a background mask (for non-editing areas). To fix the Gaussian noise tensor, In order to proceed according to the diffusion schedule A function that adds noise.
[0017] A second aspect of this invention provides a crack image generation apparatus based on a cyclic consistency constraint and a diffusion model, the apparatus comprising: The data acquisition module is used to acquire a training image set and its corresponding mask, wherein the training image set includes cracked images and crack-free images; and to acquire a target image and its corresponding mask. The encoding module is used to map the images in the training image set to a low-dimensional latent space through an encoder to obtain a training latent representation; and to map the target image to a low-dimensional latent space through an encoder to obtain a source latent representation; The mask processing module is used to downsample the mask corresponding to the training image set to obtain a training latent mask; and to downsample the mask corresponding to the target image to obtain a latent mask; A bidirectional diffusion generation framework includes a first generation process and a second generation process; the first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image based on the trained latent mask, and the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image based on the trained latent mask; The training module is used to train the bidirectional diffusion generation framework in the low-dimensional latent space using a cycle consistency constraint; wherein the cycle consistency constraint is used to ensure that the result obtained after the image undergoes successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. A generation module is configured to perform iterative denoising based on the source latent representation and the latent mask, through the target generation process determined in the trained bidirectional diffusion generation framework, to generate a target latent representation; and The decoding module is used to reconstruct the latent representation of the target into the image space through the decoder to obtain the fused crack image.
[0018] A third aspect of the present invention provides an electronic device, characterized in that it includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a crack image generation method based on a cycle consistency constraint and diffusion model.
[0019] A fourth aspect of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, is a crack image generation method based on a cycle consistency constraint and diffusion model.
[0020] This application has at least the following advantages or beneficial effects: This application combines adversarial and diffusion models, introduces a cyclic consistency constraint, and trains a denoising diffusion probability model. Then, the denoising probability model is used to fuse ordinary bare ground surfaces with crack masks to generate realistic crack images.
[0021] This application enables the intelligent generation of controllable and realistic crack datasets with small sample sizes and low cost, solving the problems of high cost and limited sample size in acquiring specific crack data. With the supplementation of generative data, the recognition and application level of the automatic crack identification algorithm model is improved, which has significant theoretical and practical implications. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart of a crack image generation method based on a cyclic consistency constraint and diffusion model proposed in an embodiment of this application; Figure 2 This is a structural diagram of a crack image generation device based on a cyclic consistency constraint and diffusion model proposed in an embodiment of this application; Figure 3 This is a schematic diagram of a training process proposed in an embodiment of this application; Figure 4 This is a schematic diagram of an electronic device according to this application. Detailed Implementation
[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] Please refer to Figure 1 , Figure 1 This is a flowchart of a crack image generation method based on a cyclic consistency constraint and diffusion model proposed in an embodiment of this application. Figure 1 As shown, a crack image generation method based on cyclic consistency constraints and a diffusion model includes: S100: Obtain a training image set and corresponding masks, wherein the training image set includes cracked images and crack-free images; and obtain the target image and corresponding mask; In this embodiment, the training image set includes cracked and crack-free images of natural bare land scenes, and is equipped with a mask corresponding to the pixel space. The mask defines the area with a value of 1 as the editable area and the area with a value of 0 as the background preservation area. The spatial resolution of the mask is consistent with that of the corresponding image. The target image is a natural bare land image to be subjected to crack synthesis or crack repair. Its matching mask also follows the above-mentioned pixel space region definition rules and matches the spatial resolution of the target image.
[0026] S200: The images in the training image set are mapped to a low-dimensional latent space through an encoder to obtain a training latent representation, and the corresponding mask is downsampled to obtain a training latent mask; and the target image is mapped to a low-dimensional latent space through an encoder to obtain a source latent representation, and the corresponding mask is downsampled to obtain a latent mask; In this embodiment, the encoder used is a frozen variational autoencoder (VAE). Deterministic encoding is performed on input images such as images in the training image set and target images. The mean of the posterior distribution of the variational autoencoder is taken as the initial latent variable, and then linear scaling and translation operations are applied to obtain the final training latent representation and source latent representation. The calculation formulas for linear scaling and translation are as follows:
[0027] in, The input image has the shape of The numerical range is ; Batch size. Number of RGB channels; The mean (without sampling) of the latent variable posterior distribution output by the VAE encoder has the following shape: For latent representation, the shape is ; The potential representation scaling factor (a constant in the implementation, by default) ); The potential representation of the translation factor (a constant in the implementation, by default) ); The spatial resolution of the input image; implement the default. The potential representation resolution is .
[0028] The potential representation will then be Restore the image to pixel space :
[0029] in, The decoded image has the following shape: The numerical range is ,in The spatial dimensions of the potential space; To freeze the decoder mapping for VAE; Input for latent space; This is the scaling / translation constant.
[0030] For the pixel space masks corresponding to the training and target images, binarization is first performed. A pointer function is used to set pixel values greater than or equal to a set threshold to 1, and the rest to 0, resulting in a binary mask. The binarization calculation formula is as follows:
[0031] in, For a binary mask, the value is [value]. ; This is an indicator function; it outputs 1 if the condition is true, and 0 otherwise.
[0032] Then, a boundary dilation operation is performed on the binary mask:
[0033] in, The mask after boundary expansion has the following shape: It is used to improve boundary tolerance and reduce the generation / repair of missing coatings. Expand the kernel size (implementing the default) ); Max pooling with a step size of 1 (preserving the maximum value within the window) is equivalent to dilating the binary mask.
[0034] Finally, the mask is downsampled to the latent space resolution. Adaptive max pooling is used to ensure that small mask details are not erased during downsampling.
[0035] in, For adaptive max pooling, downsample to the specified output size; It is a latent space mask with the shape of Implement default .
[0036] S300: Construct a bidirectional diffusion generation framework, the framework including a first generation process and a second generation process; the first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image according to the trained latent mask, and the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image according to the trained latent mask; In this embodiment, the bidirectional diffusion generation framework is constructed in a low-dimensional latent space, with a diffusion model as its core and an adversarial model structure integrated. The first generation process corresponds to the crack generator GAB, which generates the latent representation of the image from the A→B direction to the latent representation of the image with cracks. The second generation process corresponds to the repair generator GBA, which repairs the latent representation of the image from the B→A direction to the latent representation of the image without cracks. Both generators are trained using diffusion noise prediction and are integrated into the diffusion process of the latent space. The total number of diffusion steps is defined as T, and a progressive noise intensity sequence is set. ,make
[0037] Therefore, for any time step Forward diffusion (noise addition) is:
[0038] in, This represents the total number of diffusion steps. For diffusion time step ( ); For the first The noise intensity of the step; For the first The fidelity coefficient of the step; Cumulative fidelity coefficient (before) step (product of) It is a noise-free latent variable with the shape of ; For the first The noisy latent variables of the step, with the same shape It is Gaussian noise, and its shape is similar to... Same, satisfy It is the unit covariance matrix.
[0039] To ensure the generation effect of the bidirectional diffusion generation framework, a first discriminator is also configured for the framework. The first discriminator is designed for the cracked domain and is used to distinguish between true and false latent representations of cracked images synthesized in the first generation process. All discriminators are time-step conditionalized PatchGAN structures. The input is the image and the diffusion time step t, and the output is a patch-level logits graph. At the same time, a multi-dimensional hybrid loss function is configured for the framework, including diffusion noise prediction loss, cycle consistency loss, and adversarial loss based on least squares GAN. Each loss term plays a different role in training constraints.
[0040] S400: In the low-dimensional latent space, the bidirectional diffusion generation framework is trained using a cycle consistency constraint; wherein, the cycle consistency constraint is used to ensure that the result obtained after the image undergoes the successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. In this embodiment, as Figure 3 As shown, Stage 1 (supervised) pre-trained crack repair generator Stage 1 uses paired data. Supervised training repair direction generator :in For images with cracks, For the corresponding target image to be repaired, For crack masking. Its goal is to mask cracks within a given context. With mask Under certain conditions, the noise prediction capability of the learning diffusion model is improved.
[0041] In the latent space,
[0042] During training Adding noise to obtain ,enter Predicted noise and use Optimization. Therefore, the total loss for Stage 1 is:
[0043] in, Repair generator (direction) ), trained in the form of diffuse noise prediction; Image with cracks; For the corresponding target image to be repaired; For pixel space mask; For the reason The encoded "target latent"; This refers to the VAE encoding process, whose output is a latent representation. ; The optimization objective for Stage 1; Predict loss for diffused noise.
[0044] Stage 2 (Unpaired) Cyclic Consistency Self-Supervised Training of Crack Generator Stage 2: Freeze the pre-trained data. Instead, train the crack generator. Simultaneously, a discriminator is introduced at a low time step. Adversarial and perceptual constraints are used to improve the realism of the generated results in the pixel domain.
[0045] First, enable the latent space cycle consistency loss at low time steps. During the training loop, first use the frozen... For real crack samples Performing multi-step sampling yields pseudo-clean latent variables. (Details are implemented by the sampler, and each step includes a redraw truncation):
[0046] Then use As a condition Predictions in The noise on the sample is removed, and the reconstructed latent variables are obtained from the "clean sample estimate". Thus, the latent space cycle consistency loss is defined as follows:
[0047] in, For crack generator (direction) ), trained in the form of diffuse noise prediction; For real crack images The potential representation of; Frozen due to from The obtained "pseudo-clean" latent variables; This is a multi-step sampling / denoising operator (including redraw truncation), and the output is a latent variable; For the reason Estimates of latent variables of cracks obtained by reconstruction under given conditions; This represents the latent space cyclic consistency loss.
[0048] Secondly, adversarial loss (LSGAN, PatchGAN discriminator) is added at low time steps. This is for "Time-Step Conditional PatchGAN". The input is an image and a time step. The output is a patch-level logits graph:
[0049] On the generator side, use an adversarial loss of the form of least-squares GAN (LSGAN):
[0050] Use labeled smoothed loss on the discriminator side:
[0051] in, For the discriminator (for the "cracked domain") "), output patch logits; For any input image; The diffusion time step is used as a conditional input to the discriminator; The patch logits graph output by the discriminator (shape: ); This is a real crack image; Crack images generated by the generator (by latent variables) (Decoded by VAE, see below) For the generator's adversarial loss; The adversarial loss of the discriminator; An expected / mean operator for averaging over a batch (and all patch locations).
[0052] In addition, an R1 gradient penalty is introduced during discriminator training:
[0053] in, This is the R1 gradient penalty term; These are the R1 coefficients; For the input image Operators for finding the gradient; square Norm.
[0054] Finally, the total loss and low loss for Stage 2 are given. Gating. Generator in Stage 2 The overall goal is:
[0055] The adversarial / perceptual / cyclic consistency term is enabled only at low time steps and can be formalized as a gating function:
[0056] in, Total loss on the generator side for Stage 2; These are the weight hyperparameters for the corresponding loss terms; For time-step gating function; when Enable low Loss, when Skip when; This is an indicator function; if the condition inside the parentheses is true, it outputs 1; otherwise, it outputs 0. The low time step threshold (in the implementation, it is a positive integer, satisfying...) ).
[0057] In addition, the generator parameters are maintained using an exponential moving average (EMA) during training:
[0058] in, These are the parameters for the current generator; For EMA version parameters; The assignment / in-place update symbol means "update the variable on the left with the value of the expression on the right"; EMA attenuation coefficient ( (This is specified in the configuration during implementation).
[0059] S500: Based on the source latent representation and the latent mask, perform iterative denoising through the target generation process determined in the trained bidirectional diffusion generation framework to generate the target latent representation; In this embodiment, the target generation process is determined according to the actual task requirements. If it is a crack synthesis task, then the crack generator corresponding to the first generation process is selected. As a denoising network, if the task is crack repair, the repair generator corresponding to the second generation process is selected. As denoising networks, both types of tasks share the same latent space diffusion and redraw truncation mechanisms. During inference, given the input image... With mask ,get and Then select the start time step. and in Add noise and construct the sampling starting point:
[0060] Then, several steps of denoising sampling are performed, with redraw truncation applied at each step. The sampling is then complete. The output image is then obtained by decoding via a variational autoencoder (VAE).
[0061] in, For the inference start time step ( The larger the value, the stronger the starting noise. The initial noisy latent variables of the sampling chain; To fix the noise, it remains unchanged within the same sampling chain; These are the clean latent variables obtained after the sampling is completed.
[0062] To ensure non-editing areas Strict consistency is maintained throughout the entire sampling chain. Each step of the sampling / multi-step denoising process performs a redraw truncation on the latent variables, forcing the background to align with the "noisy state of the context latent at the same time step":
[0063] Among them, the "known noise state" on the background side consists of the same fixed noise. generate:
[0064] in, The updated current latent variable; The assignment / in-place update symbol means "update the variable on the left with the value of the expression on the right"; The model provides the current step prediction latent variable (or its equivalent intermediate quantity); Use as a background mask (non-editable area); when When this value is 1, the "known noise state" of the background is forced to be used; To fix the Gaussian noise tensor, the shape and Same; remains unchanged within a single sampling chain to ensure consistent background over time; In order to proceed according to the diffusion schedule A function that adds noise.
[0065] S600: The latent representation of the target is reconstructed into the image space through a decoder to obtain the fused crack image.
[0066] In this embodiment, the decoder used is a frozen variational autoencoder (VAE) decoder. The latent representation of the target in the low-dimensional latent space is reconstructed into a pixel-space image through this decoder. The reconstructed image is a tensor of shape B, 3, 8h, 8w, with a numerical range of [...]. [1,1], where h,w is the spatial size of the latent space. The reconstructed pixel spatial image is the final fused crack image. If it is a crack synthesis task, the image is a cracked image with realistic crack features fused with a crack-free background. If it is a crack repair task, the image is a crack-free image after crack repair is completed.
[0067] Please refer to Figure 2 , Figure 2 This is a structural diagram of a crack image generation device based on a cyclic consistency constraint and diffusion model, as proposed in an embodiment of this application. Figure 2 As shown in the embodiments of this disclosure, a crack image generation device based on a cyclic consistency constraint and diffusion model is also provided. The device includes: Data acquisition module 201 is used to acquire a training image set and corresponding masks, wherein the training image set includes cracked images and crack-free images; and to acquire a target image and corresponding mask. The encoding and masking processing module 202 is used to map the images in the training image set to a low-dimensional latent space through an encoder to obtain a training latent representation, and to downsample the corresponding mask to obtain a training latent mask; and to map the target image to a low-dimensional latent space through an encoder to obtain a source latent representation, and to downsample the corresponding mask to obtain a latent mask; The bidirectional diffusion generation framework 203 includes a first generation process and a second generation process; the first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image based on the trained latent mask, and the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image based on the trained latent mask. Training module 204 is used to train the bidirectional diffusion generation framework in the low-dimensional latent space using cycle consistency constraints; wherein, the cycle consistency constraints are used to ensure that the result obtained after the image undergoes successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. Generation module 205 is configured to generate a target latent representation by performing iterative denoising based on the source latent representation and the latent mask, through the target generation process determined in the trained bidirectional diffusion generation framework; and The decoding module 206 is used to reconstruct the latent representation of the target into the image space through the decoder to obtain the fused crack image.
[0068] This disclosure also provides an electronic device, please refer to... Figure 4 , Figure 4 This is a schematic diagram of an electronic device illustrated in an embodiment of this disclosure. For example... Figure 4 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus. The memory 110 stores a computer program that can run on the processor 120 to implement the steps in the crack image generation method based on the cyclic consistency constraint and diffusion model disclosed in this embodiment.
[0069] The disclosed embodiments also provide a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of a computer device, enables the computer device to perform the steps in the crack image generation method based on a cyclic consistency constraint and diffusion model as described in the embodiments of this disclosure.
[0070] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0074] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.
[0075] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0076] The above provides a detailed description of a crack image generation method based on a cyclic consistency constraint and diffusion model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for generating a crack image based on a cycle-consistency constraint and diffusion model, characterized in that, include: Obtain a training image set and its corresponding mask, wherein the training image set includes cracked images and crack-free images; The images in the training image set are mapped to a low-dimensional latent space through an encoder to obtain a training latent representation, and the corresponding mask is downsampled to obtain a training latent mask; A bidirectional diffusion generation framework is constructed, the framework including a first generation process and a second generation process; The first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image based on the trained latent mask; the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image based on the trained latent mask. In the low-dimensional latent space, the bidirectional diffusion generation framework is trained using a cycle consistency constraint; wherein, the cycle consistency constraint is used to ensure that the result obtained after the image undergoes successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. The target image and its corresponding mask are obtained. The target image is mapped to a low-dimensional latent space through an encoder to obtain the source latent representation. The corresponding mask is downsampled to obtain the latent mask. Based on the source latent representation and latent mask, iterative denoising is performed through the target generation process determined in the trained bidirectional diffusion generation framework to generate the target latent representation; The latent representation of the target is reconstructed into the image space using a decoder to obtain the fused crack image.
2. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 1, characterized in that, The training of the bidirectional diffusion generation framework is constrained by the cycle consistency loss function, which calculates the difference between the reconstruction result obtained after the image has passed through the second generation process and the first generation process in sequence, and the initial latent representation of the image when it enters the cycle.
3. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 1, characterized in that, When training the bidirectional diffusion generation framework, a first discriminator is also used; the first discriminator discriminates the potential representation of the cracked image synthesized by the first generation process and calculates the adversarial loss using the least squares GAN loss function.
4. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 3, characterized in that, The bidirectional diffusion generation framework is trained in two stages: In the first stage, the second generation process is pre-trained using a paired set of training images to optimize the diffusion model loss; In the second stage, the parameters of the second generation process are fixed, and the first generation process and the first discriminator are trained using an unpaired training image set; wherein, the optimized total loss function is: in, For Stage 2, the generator-side total loss is... To predict the loss for diffused noise, For latent space cyclic consistency loss, For the generator's adversarial loss, These are the weight hyperparameters for the corresponding loss term.
5. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 4, characterized in that, The corresponding mask is downsampled using max pooling to obtain a potential mask that matches the size of the low-dimensional potential space. in, For adaptive max pooling, downsampling is performed to the specified output size. For latent space masking, This is the mask after boundary expansion.
6. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 1, characterized in that, The process of performing iterative denoising includes: In each denoising step, the current noise latent representation, time step, source latent representation, and latent mask are input into the denoising network to obtain the predicted denoising result; The source latent representation is used to calculate the known latent representation corresponding to the current time step through a forward diffusion process; The predicted denoising result and the known latent representation are mixed element-wise according to the latent mask. The predicted denoising result is used to fill the inner region of the latent mask, and the known latent representation is used to fill the outer region of the latent mask to obtain the denoising result of the current step, which is used as the current noise latent representation for the next denoising step. Repeat the above steps until the preset number of denoising steps are completed to obtain the final latent representation of the target.
7. The crack image generation method based on cyclic consistency constraints and diffusion model according to claim 6, characterized in that, The weighted mixing operation is as follows: in, For the updated current latent variable, "Assignment / in-place update symbol, For the model to predict latent variables in the current step, Use as a background mask (for non-editing areas). To fix the Gaussian noise tensor, In order to proceed according to the diffusion schedule A function that adds noise.
8. The crack image generation device based on a cyclic consistency constraint and diffusion model according to any one of claims 1-7, characterized in that, The device includes: The data acquisition module is used to acquire a training image set and its corresponding mask, wherein the training image set includes cracked images and crack-free images; and to acquire a target image and its corresponding mask. The encoding and masking module is used to map the images in the training image set to a low-dimensional latent space through an encoder to obtain a training latent representation, and to downsample the corresponding mask to obtain a training latent mask; and to map the target image to a low-dimensional latent space through an encoder to obtain a source latent representation, and to downsample the corresponding mask to obtain a latent mask; A bidirectional diffusion generation framework includes a first generation process and a second generation process; the first generation process is used to synthesize the latent representation of a crack-free image into a latent representation of a cracked image based on the trained latent mask, and the second generation process is used to repair the latent representation of a cracked image into a latent representation of a crack-free image based on the trained latent mask; The training module is used to train the bidirectional diffusion generation framework in the low-dimensional latent space using a cycle consistency constraint; wherein the cycle consistency constraint is used to ensure that the result obtained after the image undergoes successive transformations of the second generation process and the first generation process is consistent with the initial latent representation of the image when it enters the cycle. A generation module is configured to perform iterative denoising based on the source latent representation and the latent mask, through the target generation process determined in the trained bidirectional diffusion generation framework, to generate a target latent representation; and The decoding module is used to reconstruct the latent representation of the target into the image space through the decoder to obtain the fused crack image.
9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the crack image generation method based on the cyclic consistency constraint and diffusion model as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the crack image generation method based on a cyclic consistency constraint and diffusion model as described in any one of claims 1 to 7.