A method, apparatus and electronic device for crack data augmentation

Abstract

This application provides a method, apparatus, and electronic device for augmenting crack data, relating to the field of data processing. In this method, textual semantic embedding vectors are generated by parsing crack attribute description text, and multi-stage noise addition is performed on the original crack image based on cosine noise scheduling to construct a noisy image sequence. The noisy image sequence, textual semantic embedding vectors, and time-step index are input into a conditional U-Net denoising network, and multi-layer semantic features of the image are extracted and fused through a multi-scale convolutional structure and a text-image cross-attention structure. A crack-aware dynamic noise scheduling strategy is constructed based on the crack area ratio, and noise parameters are adaptively adjusted to complete network training. Finally, in the inference stage, random noise is used as the starting point, and back-diffusion is performed based on the textual semantic embedding vectors to generate a crack augmented image that is consistent with the text description and has realistic texture and adaptability to complex backgrounds. Implementing the technical solution provided in this application facilitates the improvement of the accuracy of crack augmented image generation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method, apparatus, and electronic device for augmenting crack data. Background Technology

[0002] With the increasing demand for infrastructure inspection and maintenance, computer vision-based automatic crack identification technology for roads and bridges has been extensively studied. In these applications, deep learning models require a large amount of high-quality, labeled crack image data for training to achieve accurate identification and segmentation.

[0003] However, traditional data augmentation methods mainly rely on image-level operations such as geometric transformations, brightness perturbations, color perturbations, and random occlusion. These methods only generate different variations based on the original image content, and the generated results still retain the original crack's morphology and texture features, failing to create new crack types that exceed the distribution of the original image. For cracks, which have complex morphological changes, are sensitive to texture details, and are highly dependent on the background, traditional augmentation methods cannot construct realistic crack structures, nor can they simulate the multi-scale cracks, intersecting cracks, fine cracks, and crack features under complex material backgrounds that exist in real-world scenes. This results in insufficient diversity of model training data, limiting the crack recognition model's ability to generalize to real-world complex scenes, and ultimately leading to low accuracy in the generated crack augmented images.

[0004] Therefore, there is an urgent need for a method, device, and electronic equipment for augmenting crack data. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for augmenting crack data, which facilitates the improvement of the accuracy of crack augmentation image generation.

[0006] The first aspect of this application provides a crack data augmentation method, the method comprising: acquiring crack attribute description text and an original crack image; generating a text semantic embedding vector by semantic parsing and sequence encoding of the crack attribute description text; performing multi-stage noise addition processing on the original crack image based on cosine noise scheduling to generate a noisy image sequence covering different noise intensities; inputting the noisy image sequence, the text semantic embedding vector, and the corresponding time step index into a conditional U-Net denoising network, and extracting multi-layer semantic features of the image through a multi-scale convolutional structure, wherein the multi-layer semantic features of the image are used to characterize crack structural features and background texture features; constructing a text-image cross-attention structure in the bottleneck layer of the conditional U-Net denoising network, and through... The text-image cross-attention structure integrates the text semantic embedding vector and the multi-layer semantic features of the image in both spatial and semantic dimensions, making the backdiffusion process constrained by text semantics. Based on the crack area ratio of the original crack image, a crack-aware dynamic noise scheduling strategy is constructed, and the noise parameters at the corresponding time steps are adjusted according to the crack-aware dynamic noise scheduling strategy to train the conditional U-Net denoising network based on the noise parameters. In the inference phase, the trained conditional U-Net denoising network uses random noise as the initial image, and inputs the random noise and the text semantic embedding vector into the backdiffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has the ability to adapt to realistic textures and complex backgrounds.

[0007] A second aspect of this application provides a crack data augmentation apparatus, comprising an acquisition module and a processing module. The acquisition module is used to acquire crack attribute description text and an original crack image, and to generate a text semantic embedding vector by semantic parsing and sequence encoding of the crack attribute description text. The processing module is used to perform multi-stage noise addition processing on the original crack image based on cosine noise scheduling to generate a noisy image sequence covering different noise intensities. The processing module is further used to input the noisy image sequence, the text semantic embedding vector, and the corresponding time step index into a conditional U-Net denoising network, and to extract multi-layer semantic features of the image through a multi-scale convolutional structure. The multi-layer semantic features of the image are used to characterize crack structural features and background texture features. The processing module is further used to construct a bottleneck layer in the conditional U-Net denoising network. The text-image cross-attention structure fuses the text semantic embedding vector and the multi-layer semantic features of the image in both spatial and semantic dimensions, making the backdiffusion process constrained by text semantics. The processing module is further configured to construct a crack-aware dynamic noise scheduling strategy based on the crack area ratio of the original crack image, and adjust the noise parameters at corresponding time steps according to the crack-aware dynamic noise scheduling strategy to train the conditional U-Net denoising network based on the noise parameters. The processing module is also configured to use random noise as the initial image during the inference phase of the trained conditional U-Net denoising network, and input the random noise and the text semantic embedding vector together into the backdiffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has realistic texture and adaptability to complex backgrounds.

[0008] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method described above.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described above.

[0010] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: By acquiring crack attribute description text and generating text semantic embedding vectors, the crack generation process is transformed from a traditional random generation mode to a conditional generation mode with explicit semantic control. This enables controllable synthesis of crack width, direction, orientation, morphology, and material, ensuring consistency between the augmented image and the target description. Multi-stage noisy image sequences are constructed using cosine noise scheduling, allowing the model to learn the layer-by-layer recovery rules of crack structures under different noise intensities. This improves the stability and refinement of the denoising network in complex structure reconstruction, providing a foundation for high-fidelity crack generation. Multi-scale convolutional structures extract multi-layer semantic features from the image, enabling deep features to simultaneously capture crack structure patterns, crack texture details, and background material textures. This achieves structured modeling of complex scenes, providing complete semantic support for subsequent semantic binding and denoising reconstruction.

[0011] By constructing a text-image cross-attention structure at the bottleneck layer, the crack generation process is constrained by textual attributes in both spatial and semantic dimensions, ensuring strict semantic consistency in the back-diffusion process and avoiding mismatches between the generated crack morphology and the expected description. By introducing a crack-aware dynamic noise scheduling strategy driven by the crack area ratio, the structure of fine and micro-cracks is not lost in the early noise addition stage, enabling the network to more stably recover the subtle textures of small-scale damage, improving the quality of small-target crack generation, and providing support for highly sensitive crack scenarios in engineering practice. By using random noise as the initial image and textual semantics as the generation condition during the inference stage, the model can progressively construct a complete crack image from unstructured input. This ensures that the final generated crack augmented image possesses realism, completeness, and controllability in terms of structural morphology, texture details, and background adaptation, significantly improving the accuracy of crack data augmented image generation. Attached Figure Description

[0012] Figure 1 A flowchart illustrating a crack data augmentation method provided in an embodiment of this application; Figure 2 A schematic diagram of a crack data augmentation device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0013] Explanation of reference numerals in the attached figures: 21. Acquisition module; 22. Processing module; 31. Processor; 32. Communication bus; 33. User interface; 34. Network interface; 35. Memory. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0015] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0016] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0017] To address the aforementioned technical problems, this application provides a crack data augmentation method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating a crack data augmentation method provided in an embodiment of this application. The method is applied to a server and includes steps S110 to S160, as follows:

[0018] S110. Obtain the crack attribute description text and the original crack image, and generate a text semantic embedding vector by semantic parsing and sequence encoding of the crack attribute description text.

[0019] Specifically, crack attribute description text refers to textual content recorded in natural language to describe the appearance and scene information of cracks. This textual content typically includes attributes such as crack width, crack direction, crack orientation, crack morphology, and crack material. For example, in a road inspection scenario, the crack attribute description text could describe a longitudinal through crack about one millimeter wide on the concrete surface of a bridge deck, or multiple fine network cracks with localized water seepage appearing on the inner lining of a tunnel. The text describing crack width, crack direction, and crack morphology are all components of the crack attribute description text, forming the input source for the subsequent semantic modeling process.

[0020] Raw crack images refer to real crack images captured by cameras, drones, or fixed inspection equipment. These images have not undergone any generative processing and directly reflect the actual morphology and background characteristics of the cracks in the engineering structure. For example, in asphalt pavement acquisition, raw crack images may show longitudinal cracks, transverse cracks, or crazing textures, and include background information such as tire tracks, shadows, and lighting variations. In bridge web inspection, raw crack images may present a complex scene where concrete cracks coexist with interfering factors such as rust and water stains. Raw crack images form the basis for subsequent noise addition processing and backdiffusion generation.

[0021] Semantic parsing refers to the process of identifying and extracting semantic information related to crack features from crack attribute description text. This includes identifying terms related to crack width, crack direction, crack trajectory, crack morphology, and crack material, and establishing the contextual relationships between these terms. For example, the term "approximately one millimeter wide" might be identified as crack width semantics; "vertical penetration" might be identified as a combination of crack direction and crack trajectory semantics; "concrete surface" might be identified as crack material semantics; and "networked distribution" might be identified as crack morphology semantics. The role of semantic parsing is to transform natural language into structured semantics, enabling the model to understand the engineering implications of crack descriptions.

[0022] Sequence encoding refers to the process of organizing semantically parsed text content into a discrete sequence of tokens and converting these tokens into a vector sequence that can be processed by deep learning models. In sequence encoding, the text describing crack attributes first undergoes word segmentation, breaking the entire text down into tokens such as bridge surface, concrete, surface, existence, a line, approximately wide, one millimeter, longitudinal, through, crack, mesh, and peeling. Then, each token is mapped to a vector, enabling the model to process semantic information in vector form. The purpose of sequence encoding is to transform natural language text into a serialized semantic structure that the model can recognize and compute.

[0023] A text semantic embedding vector refers to a fixed-dimensional vector obtained by inputting a sequence of encoded vectors into a semantic modeling network. This vector represents the overall text describing crack attributes. It integrates crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics, and crack material semantics, preserving the relationships between these semantics. For example, when the text describes a longitudinal through-crack approximately one millimeter wide on a concrete surface, the text semantic embedding vector will simultaneously express, in compressed vector form, comprehensive information such as the crack's presence in the concrete material, its one-millimeter width, its longitudinal extension, and its overall continuous distribution. This text semantic embedding vector ultimately serves as a conditional input to the diffusion model, controlling the consistency of the crack image's morphology, orientation, and background during the generation stage, ensuring the generated result matches the meaning of the original text description.

[0024] Furthermore, the crack attribute description text is first processed using text regularization, word boundary parsing, and semantic tagging to form a structured text sequence. Text regularization unifies full-width characters, uppercase and lowercase letters, redundant whitespace, and special symbols in the original text into a standardized form. For example, different writing styles of millimeters and cm are reduced to a unified width description word, and redundant spaces and line breaks are deleted or merged. Word boundary parsing uses a word segmentation algorithm to segment stable word units at the character level, identifying words related to crack width, crack direction, crack morphology, and crack material. Semantic tagging adds semantic category labels to each word or phrase. For example, approximately one millimeter wide is labeled as crack width semantics, along the bridge length direction as crack direction semantics, network cracks as crack morphology semantics, and concrete surface as crack material semantics. This transforms the unstructured original sentence into a structured text sequence arranged chronologically, with each position carrying both word form and semantic category information. This process can be described as mapping the original text to a standardized text sequence, as follows:

[0025]

[0026] in, This represents the original crack attribute description text. This represents a structured text sequence after text regularization, word boundary parsing, and semantic tagging. This represents a text processing function that comprehensively performs regularization, word segmentation, and annotation. This function converts non-standard, noisy natural language descriptions into structured representations that can be stably processed by subsequent coding models.

[0027] After obtaining the structured text sequence, it is input into a text encoding model, and a word vector sequence is generated through the word embedding layer of the text encoding model. Specifically, each text tag in the structured text sequence is mapped to an index, and a corresponding high-dimensional dense vector is found for each index in the word embedding layer. This makes words with similar semantics closer together in the vector space, and words with unrelated semantics farther apart. For a length of... Structured text sequence The corresponding word vector sequence is obtained through the word vector embedding layer. The process can be described as follows:

[0028]

[0029] in, Indicates the position of a structured text sequence The text mark at the location, This represents the word vector corresponding to the text tag. This represents the word-to-vector mapping function implemented by the word embedding layer. This represents the length of the structured text sequence; through this mapping, the originally discrete word symbols are converted into differentiable vector representations, enabling subsequent networks to learn the semantic relationships in the crack attribute descriptions through vector operations.

[0030] After generating the word vector sequence, the sequence is input into a semantic parsing network structure. A multi-layer self-attention mechanism is used to capture the contextual relationships between crack width semantics, crack direction semantics, crack trajectory semantics, crack morphology semantics, and crack material semantics. Specifically, in each self-attention layer, query vectors, key vectors, and value vectors are constructed based on the word vector sequence. Relevance weights between words at different positions are calculated using vector similarity, and these weights are used to perform a weighted combination on the value vectors, resulting in a new semantic representation that considers the global context. For the first... In the layer semantic parsing network, the first The context representation of a location can be described as follows:

[0031]

[0032] in, Indicates the first Layer semantic parsing network at location Contextual semantic representation at the location, Indicates the first Mid-layer position Position Attention weights Indicates the first Mid-layer position The value vector, Indicates sequence length; attention weights The similarity normalization results from the query vector and key vector give greater weight to semantically related word pairs. Through this process, the semantic parsing network can automatically learn the dependencies between semantics such as approximately one millimeter wide and crack, along the length of the bridge and through, and mesh and crack. This allows for the explicit encoding of the contextual associations between crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics, and crack material semantics in a high-dimensional representation space.

[0033] After modeling the contextual associations using a multi-layer self-attention mechanism, a global sequence compression operation is performed on these contextual associations in the sequence encoding network to generate text semantic embedding vectors. Specifically, the contextual semantic representation sequence output by the last layer of the semantic parsing network is used... The input sequence encoding network aggregates the information of the entire sequence into a fixed-dimensional vector through global pooling or global projection. This vector represents the semantic content of the text describing crack attributes. For example, an average aggregation method can be used to weight and summarize the semantic representations of all positions to obtain the text semantic embedding vector. ,as follows:

[0034]

[0035] in, This indicates that the last layer of the semantic parsing network is at position Contextual semantic representation, Indicates the length of the structured text sequence. This represents the final text semantic embedding vector. Through this global sequence compression operation, the crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics, and crack material semantics scattered at various word positions, as well as their contextual relationships, are uniformly encoded into a fixed-dimensional vector. This vector not only retains the overall meaning of the crack attribute description text, but also has the compact expressive ability to be directly used as a conditional feature to participate in the subsequent diffusion generation and denoising process.

[0036] S120. Perform multi-stage noise addition processing on the original crack image based on cosine noise scheduling to generate a noisy image sequence covering different noise intensities.

[0037] Specifically, cosine noise scheduling refers to setting the noise intensity at each time step during the noise addition process. Instead of using a simple linear increase or fixed increment, it uses the variation curve of a cosine function as a reference, designing the entire noise intensity change over time as a smooth trajectory similar to a cosine curve. This results in different noise intensity distributions in the early, middle, and late stages of noise addition. The advantage of using cosine noise scheduling is that it can maintain a low noise intensity in the early stages of noise addition, preserving as much detail as possible in the original crack image. In the middle and later stages, the noise intensity is gradually increased, allowing the image to gradually approach a high-noise state. This is beneficial for the diffusion model to learn its ability to recover crack structures from different noise levels during training. For example, in a noise addition process from the start time step to the end time step, cosine noise scheduling can be represented as a smooth curve with very slow noise changes in the first few steps, faster noise increases in the middle, and saturation in the last few steps. This scheduling method is called cosine noise scheduling.

[0038] Multi-stage noise addition refers to adding a large amount of noise to the original crack image rather than all at once. Instead, noise is added gradually over a series of discrete time steps, with each time step corresponding to a noise intensity value. Each noise addition results in a new noisy image, creating multiple stages from low to high noise. These stages can be understood as different noise levels. For example, the first stage adds only a very small amount of noise, making the crack morphology and background texture almost completely discernible; the middle stage adds a moderate level of noise, leaving the crack outline still visible but with local details becoming blurred; the later stage adds a high level of noise, significantly obscuring the crack morphology and background texture, resulting in a nearly random texture distribution. By adding noise layer by layer over multiple time steps, multi-stage noise addition allows the model to learn how to recover the crack structure at different noise levels.

[0039] Noise intensity refers to the proportion or energy of noise added to the original crack image at each noisy time step, used to measure the relative abundance of noise components in the current image compared to the original image components. At low noise intensity, the noisy image is still primarily dominated by the content of the original crack image; crack edges, crack direction, and background texture remain clear. At medium noise intensity, the crack outline in the noisy image begins to be perturbed by random textures, blurring local details, but the overall structure is still discernible. At high noise intensity, random textures dominate the noisy image, and the crack is almost completely submerged in noise, retaining only a very small number of recoverable structural clues. Noise intensity is allocated in conjunction with cosine noise scheduling, ensuring that each time step corresponds to a reasonable noise level.

[0040] A noisy image sequence refers to an ordered set of all intermediate images saved in chronological order during the multi-stage noise addition process on the original crack image. Each image corresponds to a specific noise intensity and time step index. The first image in the noisy image sequence is very close to the original crack image, with only minimal noise added; several intermediate images correspond to different levels of noise perturbation; and the last image corresponds to the highest noise intensity, approaching a completely noisy state. For example, in a noise addition process with one hundred time steps, one hundred noisy images can be obtained. Arranging these one hundred images in chronological order constitutes a noisy image sequence covering noise levels from low to high. This noisy image sequence is used as input for backdiffusion during the training phase, enabling the denoising network to learn how to gradually recover the crack structure from images with different noise intensities, thereby supporting the generation of subsequent high-quality crack augmentation images.

[0041] Furthermore, a continuous time-step sequence is first constructed based on cosine noise scheduling, and a noise intensity ratio determined by the cosine variation trend is assigned to each time step. The total number of time steps is set to a discrete integer set, and each time step is mapped to a normalized time parameter after nonlinear stretching and perturbation correction. Then, the noise intensity ratio is determined according to a cosine function with power amplification and amplitude modulation, so that the noise intensity exhibits a piecewise characteristic on the time axis: a gradual increase in the early stage, a rapid change in the middle stage, and a slow convergence in the later stage. The time step index can be first normalized into an intermediate variable and nonlinear transformation and periodic perturbation terms can be introduced. For example, the normalized time variable can be defined as:

[0042]

[0043] Then, the normalized time variable is mapped to a noise intensity ratio, as follows:

[0044] in, This represents the current time step index, which is one to... Integers between; This represents the total number of time steps, which is a preset positive integer. This indicates a time shift offset, used to avoid overly steep changes at the starting position; This represents the nonlinear stretching coefficient, used to control the degree of compression in the early and later time periods; This represents the amplitude coefficient of the sinusoidal perturbation, used to superimpose high-frequency perturbations onto the overall cosine trend to increase the flexibility of the noise intensity curve; This represents the power compression or amplification factor, used to enhance the weight of the low-noise or high-noise phase. This represents the normalized time parameter after nonlinear mapping and perturbation correction, and its value is usually controlled to be around zero to one. Indicates time step The corresponding noise intensity ratio, with a value ranging from zero to one, is used to determine the mixing weight of the original crack image content and the random noise field at that time step.

[0045] After calculating the noise intensity ratio at each time step, a pixel-level noise mixing operation is performed on the original crack image at each time step according to the corresponding noise intensity ratio. This allows the content of the original crack image to be proportionally fused with the random noise field to generate a noisy image. Specifically, for any pixel location and any color channel in the original crack image, the noise value at the same location and in the same channel is first extracted from the random noise field corresponding to that time step. Then, the two are mixed based on a fusion model with adaptive power-law weights and normalization factors for each channel, allowing the effective information ratio and noise ratio for different time steps and different channels to be adaptively adjusted. The following pixel-level fusion method can be adopted:

[0046]

[0047] in, Indicates at time step Time, pixel position and color channels The pixel values ​​of the noisy image; This indicates the pixel location of the original crack image. and color channels Pixel values ​​on; Indicates at time step Time, pixel position and color channels The noise values ​​extracted from the random noise field are usually generated according to a zero-mean random distribution; Indicates time step The corresponding noise intensity ratio is the result obtained in the previous paragraph; Indicates targeting color channels The adaptive power coefficients of the channels are used to differentiate the texture sensitivity of different channels. For example, a uniform setting can be used in grayscale single-channel scenes, while different power parameters can be set for the luminance and chrominance channels in color images. The normalization term in the denominator ensures that the merged pixel values ​​remain stable in numerical amplitude, preventing excessive amplification or compression due to power operations. Through the above mixing method, in a relatively small... At a given moment, the noisy image is closer to the original crack image, at a larger time. At any given moment, noisy images tend to exhibit more characteristics of random noise fields.

[0048] After pixel-level noise mixing at each time step, the noisy images generated at each time step are sequentially stored into a multi-level sequence to form a noisy image sequence. Specifically, for the sequence from the first time step to the second... All noisy results at each time step are treated as high-dimensional tensors, and organized into an ordered sequence according to the time step order. Simultaneously, the time step index and noise intensity ratio are recorded as auxiliary information to allow for targeted sampling of specific noise levels during training of the Conditional U-Net denoising network. By storing the noisy images and corresponding noise intensity information from all time steps into a single noisy image sequence, selective sampling from images with different noise levels can be performed during subsequent training. This ensures that the Conditional U-Net denoising network learns the mapping relationship between crack structure restoration and background texture reconstruction across the entire noise intensity range, thus providing a stable and rich training sample base for the backdiffusion process.

[0049] S130. Input the noisy image sequence, text semantic embedding vector and corresponding time step index into the conditional U-Net denoising network, and extract multi-layer semantic features of the image through a multi-scale convolutional structure. The multi-layer semantic features of the image are used to characterize crack structure features and background texture features.

[0050] Specifically, the time step index refers to the sequence number assigned to each noisy image in a noisy image sequence. This sequence number reflects the image's position in the noise addition process and indirectly reflects the noise intensity level. Typically, the time step index is numbered sequentially from the first step to the last. A smaller time step index corresponds to a noisy image with weaker noise and more structural information, while a larger time step index corresponds to a noisy image with stronger noise and severely obscured structural information. For example, with a total of one hundred time steps, a time step index of ten represents the early, mild noise stage, and a time step index of ninety represents the later, heavily noisy stage. Inputting the time step index into the network allows the network to employ different denoising strategies based on different noise stages.

[0051] Conditional U-Net denoising network refers to a deep neural network that introduces conditional information into the traditional U-shaped encoder-decoder structure. This conditional information mainly includes text semantic embedding vectors and temporal step indices. The U-shaped structure typically consists of a progressively downsampled encoding path and a progressively upsampled decoding path, with cross-layer skip connections fusing shallow and deep features. This is very common in image restoration and generation tasks. Building upon this structure, Conditional U-Net integrates text semantic embedding vectors and temporal step indices into the feature map. This allows the network to consider not only the pixel patterns of the image itself but also the semantic attributes of cracks and the current noise stage during denoising. For example, given the same noisy image input, if the text semantic embedding vector describes vertical fine cracks, Conditional U-Net tends to recover elongated linear structures; if the text semantic embedding vector describes mesh-like cracks, the network tends to generate interwoven crack structures.

[0052] Multi-scale convolutional structure refers to the structural arrangement within the Conditional U-Net denoising network that extracts features at different spatial scales through multiple layers of convolution, downsampling, and upsampling. This allows the network to capture both local details and perceive global structure. In the downsampling path, convolutional layers progressively reduce the feature map resolution and increase the number of channels to extract large-scale structural patterns, such as the direction of a crack or its overall distribution on a component. In the upsampling path, convolutional layers progressively restore resolution and fuse shallow features through skip connections to preserve detailed information, such as the jagged edges of cracks or texture variations within cracks. Under the influence of this multi-scale convolutional structure, the network can simultaneously understand the texture variations of a crack within its local pixel neighborhood and its topological morphology across the entire image.

[0053] Image multi-layer semantic features refer to a set of feature representations formed at different network depths after multi-scale convolutional structures have undergone multi-layer processing. Each feature map corresponds to a semantic level. For example, shallow feature maps emphasize edges, textures, and details; mid-layer feature maps emphasize region patterns and local structures; and deep feature maps emphasize overall shape and global layout. For crack scenes, shallow features in image multi-layer semantic features can characterize the clarity of crack boundaries and the brightness contrast between cracks and the background; mid-layer features can characterize the branching structure of cracks and the interweaving between cracks and material textures; and deep features can characterize the distribution of cracks throughout the component, such as a single through crack, multiple parallel cracks, or complex network cracks.

[0054] Crack structure features refer to information specifically used in multi-layer semantic features of an image to characterize the geometry and spatial layout of cracks. This includes the crack's start and end points, direction, curvature, branching structure, and connections between cracks. For example, in a road image, if the crack structure features show a long, thin, high-response region extending from the bottom to the top in the image center, this corresponds to a longitudinal crack; if intersecting high-response regions form in the middle of the image, this corresponds to an intersecting crack or a cross-shaped crack. These features help the network preserve or reconstruct the overall morphology of cracks during denoising and generation.

[0055] Background texture features refer to the information used in multi-layer semantic features of an image to characterize the texture patterns of the material surface where the crack is located and the scene environment. This includes the roughness of concrete, the granularity of asphalt, the striped structure of brick walls, the reflective texture of metal surfaces, and non-structural details such as pollution, water stains, and shadows. For example, in a concrete bridge scene, background texture features can reflect the particle distribution of cement and aggregate, and the fine textures formed by the pouring direction; in an asphalt pavement scene, background texture features can reflect the irregular distribution of asphalt particles and ruts. The Conditional U-Net denoising network simultaneously models crack structural features and background texture features. During denoising and generation, it can both restore the crack morphology that conforms to the semantic description and allow the crack to be naturally embedded in the real background, thus generating crack images with a high overall realism.

[0056] Furthermore, in specific implementation, the noisy image sequence is first mapped to a base feature map, and the time step index is encoded into a time step feature vector. Then, through feature broadcasting, the time step feature vector is fused with the base feature map to obtain a time step fused feature map. For time steps of... For a noisy image, let the original pixel tensor be denoted as . The basic feature map is obtained through convolutional mapping and channel mapping. At the same time, index the time step. Encoded as time-step feature vectors ,Will The temporal-step fusion feature map is obtained by expanding the vector into a channel modulation vector through linear transformation and broadcasting it in the spatial dimension. , can be represented as:

[0057]

[0058] in, This indicates the spatial location of the time-step fused feature map. The integral channel feature vector at the location; Indicates time step Noisy images in the convolutional receptive field The neighboring pixel block within; This represents the operation of expanding a neighborhood pixel block into a column vector; This represents the weight matrix that maps neighboring pixels to the underlying feature channel space. This represents the corresponding bias vector; Indicates the time step index. This represents the time-coded vector output by the time-coding function, for example, containing... , And combinations of polynomial terms; This represents the weight matrix that maps time coding to channel modulation space; This represents a combined mapping with nonlinear activation and normalization; through this structure, the time step index is obtained via... and It is converted into a channel offset shared across all spatial locations, thus in The noise stage information is explicitly injected into the time step fusion feature map to distinguish different noise intensity stages.

[0059] Subsequently, the text semantic embedding vector is mapped to a semantic feature vector, and the semantic feature vector is then concatenated or additively fused with the base feature map to form a semantic fusion feature map. Let the text semantic embedding vector be... Semantic feature vectors are obtained through two or more layers of nonlinear mapping. Then, through feature broadcasting, it is expanded in the spatial dimension into a semantic feature map with the same spatial size as the base feature map, and then fused with the base feature map in the channel dimension. When using a combination of channel splicing and gated additive fusion, in terms of spatial location... Semantic fusion feature vector at the location It can be described as:

[0060]

[0061] in, Indicates the spatial location of the semantic fusion feature map The integral channel feature vector at the location; This represents the vector obtained by concatenating the time-step fusion feature vector and the semantic feature vector along the channel dimension. This represents a gated generation mapping for the time-step fused feature channels, used to output the... A dimensionally consistent gating coefficient vector; This represents a remapping of semantic feature vectors, used to generate... Output the matching semantic modulation vector; Represents element-wise multiplication; through gating terms It can inject semantic information on demand for different spatial locations, so that the semantic fusion feature map can simultaneously encode local image features and global crack attribute semantics, thereby maintaining unified constraints on crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics and crack material semantics in subsequent convolution processing.

[0062] After constructing the temporal-step fusion feature map and the semantic fusion feature map, these maps are sequentially input into a multi-level downsampling convolutional block of a conditional U-shaped network structure to extract low-level and high-level semantic information. The multi-level downsampling convolutional block consists of several convolutional layers, nonlinear activation layers, and downsampling operators cascaded together. Each level reduces spatial resolution while increasing the number of channels and the receptive field, forming a multi-scale image representation from shallow to deep. The multi-level downsampling process can be abstracted as a superposition of a set of linear-nonlinear transformations in the feature space, encoding features layer by layer under both temporal and semantic conditions. Let... This represents the initial feature tensor input to the encoding path after semantic fusion. Indicates the first The feature tensor output by the level downsampled convolutional block is then... In level coding, a compact expression with a channel mixing matrix and a spatial downsampling operator can be used, as follows:

[0063]

[0064] in, Indicates time step In the The feature tensor output by the subsampled convolutional block; Represents the feature tensor of the previous layer; Indicates the first Level convolution kernel tensor, This indicates a multi-channel convolution operation; Indicates the first Linear representations of step-downsampling or pooling operators are used to reduce the spatial size from... Map to smaller ; This represents the channel mixing matrix, used to perform linear reprojection in the channel space of the convolution output, thereby enhancing feature representation capabilities. Indicates the first A combination of nonlinear activation and normalization functions at the level; through this structure, shallow layers More edge and texture details are preserved, in greater depth This allows for the encoding of the overall direction of the cracks, the distribution pattern of the cracks, and the macroscopic texture of the background, thus forming image features that are progressively abstracted from low-level semantic information to high-level semantic information.

[0065] Finally, spatial resolution is restored through multi-level upsampling convolutional blocks of a conditional U-shaped denoising network, and cross-layer feature concatenation is performed. This combines low-level and high-level semantic information to generate multi-layer semantic features for the image. The multi-level upsampling convolutional blocks progressively amplify the spatial resolution, starting from the deepest feature layer. Simultaneously, at each layer, the current upsampled feature is concatenated with the corresponding downsampled feature from the encoding path, and then local details and global structure are fused through convolutional mapping. Let... This represents the initial characteristics of the decoding path. Indicates the first The features obtained from level upsampling and cross-layer concatenation are then... In the process of upsampling and fusion, the following forms can be adopted:

[0066]

[0067] in, Indicates the first The feature tensor obtained after level upsampling has a spatial size higher than and The space dimensions are matched; Indicates the first Level space upsampling operator, used to... Spatial resolution from Upgraded to It can be a combination of interpolation plus convolution or transpose convolution; Indicates the encoding path features The channel transformation operator is used to rearrange or compress channels before splicing; This represents a concatenation operation along the channel dimension, merging the upsampled features with the corresponding encoded features; Indicates the first The convolution, activation, and normalization are combined and mapped in the upsampling convolutional block. Through multi-level upsampling and cross-layer concatenation from the deepest to the shallowest layer, the high-resolution feature tensor obtained in the shallowest layer simultaneously preserves the crack edge details from the shallow layer, the local structural patterns from the middle layer, and the overall crack layout and background texture distribution from the deep layer. This high-resolution feature tensor can be regarded as the multi-layer semantic feature of the image, used to characterize the crack structure features and background texture features, and provides a complete and hierarchical feature foundation for subsequent text-guided denoising reconstruction and crack augmentation image generation.

[0068] S140. Construct a text-image cross-attention structure in the bottleneck layer of the conditional U-Net denoising network. By integrating text semantic embedding vectors and multi-layer semantic features of images in both spatial and semantic dimensions through the text-image cross-attention structure, the back diffusion process is constrained by text semantics.

[0069] Specifically, firstly, the multi-layer semantic feature tensor of the image is obtained in the bottleneck layer of the Conditional U-Net denoising network. This tensor is then unfolded from a spatial grid into image sequence features, and the text semantic embedding vector is mapped to text sequence features with the same channel dimension, ensuring that the image sequence features and text sequence features maintain a fusionable relationship in the channel dimension. Let the multi-layer semantic features of the bottleneck layer image be a tensor. In spatial location The eigenvector at point is denoted as Mapping two-dimensional grid indices to one-dimensional position indices Through position encoding vector The image sequence features are constructed using the channel reprojection matrix, denoted as:

[0070]

[0071] in, Indicates position The corresponding image sequence feature vector, This represents the channel mapping matrix used to project the bottleneck layer channel space onto the unified sequence feature space. This represents the projection matrix used to map the positional encoding vector onto the sequence feature space. Indication of spatial location Aligned positional encoding vectors are used to explicitly distinguish semantic roles at different spatial locations. For the text side, let the text semantic embedding vector be... The fused text semantic core vector is obtained through two layers of residual nonlinear mapping. And combined with learnable text position coefficients, text sequence features are generated as follows:

[0072]

[0073]

[0074] in, and These represent the semantic projection matrices of the first and second layers, respectively. This represents the second-layer bias vector. This represents a mapping function with non-linear activation, such as a piecewise linear or smooth activation function with normalization. This represents the residual mapping matrix, used to preserve information about the semantic embedding of the original text. Indicates the first in the text sequence The text sequence feature vector at each position, This represents projecting the text latent space onto a projection matrix with the same dimensions as the image sequence features. Indicates the first A learnable position embedding matrix for each text location. This represents the basis vector corresponding to that position; through the above transformation, the image sequence features are... Text sequence features Dimensional alignment enables fine-grained semantic interactions in subsequent cross-attention calculations.

[0075] After obtaining image sequence features and text sequence features, the image sequence features are constructed into a query vector, and the text sequence features are constructed into key vectors and value vectors. These are then input into a text-image cross-attention structure. By introducing higher-order association terms and channel correlation terms, the semantic association weights between the query vector and the key vector are calculated, thereby generating more expressive semantic response features. For the first... The image location and the first For each text position, construct the query, key, and value vectors as follows:

[0076]

[0077] Attention scores are constructed by combining first-order and second-order correlations, and attention weights are obtained through normalization, as follows:

[0078]

[0079] In the above relationship, This represents the query vector, key vector, and value vector in the attention subspace. These represent the query mapping matrix, the key mapping matrix, and the value mapping matrix, respectively. This represents an element-wise nonlinear compression function used to enhance the ability to distinguish highly similar regions; This represents element-wise multiplication, used to construct first-order interactive features along the channel dimension; Indicates the relationship with the first The channel selection vector corresponding to each image location is used to assign different weights to the correlation of different channels; Let represent the symmetric matrix used to construct the similarity of quadratic forms, such that Provides a high-order semantic matching degree metric; This represents the normalization operation along the text position dimension, which normalizes the scores of all text positions corresponding to each image position into a probability distribution; Indicates the first The image location is the first Attention weights for each text location; This represents a channel diagonal scaling operator that adjusts the components of a vector according to a learnable scaling factor, thereby introducing a modulation term related to the query vector into the value vector; Indicates the location in the image. The semantic response features obtained by aggregating all text semantics include both the overall semantics of the text and the crack attribute information that is most important to the current spatial location, such as fine cracks, longitudinal direction, mesh-like shape or specific material scene.

[0080] After obtaining the semantic response features, these features are reconstructed into a spatial feature map according to the bottleneck layer spatial layout. This map is then fused with the multi-layer semantic features of the original image through the execution channel, ensuring that the denoising and reconstruction operations at each time step during the backdiffusion process are continuously constrained by the textual semantics in both spatial and semantic dimensions. (The last sentence appears to be incomplete and possibly refers to a different topic.) According to the index Mapping back to the spatial grid yields the semantic response feature map. In spatial location The vector at point is denoted as Then, combined with the multi-layer semantic features of the bottleneck layer image. High-order channel fusion is performed, and the final fused features are constructed as follows:

[0081]

[0082]

[0083] in, Indicates spatial location The intermediate fusion vector is obtained by linearly combining semantic response features and original image features. This represents the mapping matrix that projects semantic response features onto the spatial fusion channel. This represents a mapping matrix that projects the multi-layer semantic features of the original image onto the same fusion channel space; Represents the spatial location of multi-layer semantic features in the original image of the bottleneck layer. The vector at that location; This represents the bottleneck layer features after fusion, where both the original image semantics and the text guidance semantics are encoded simultaneously. This represents vector concatenation along the channel dimension, which combines the original features, semantic fusion features, and their element-wise interaction features as inputs to subsequent mappings; This represents a channel hybrid mapping function composed of several fully connected layers, nonlinear activations, and normalization, used to redistribute the importance of each feature subspace within a high-dimensional channel space. Through the above reconstruction and fusion, the resulting... The upsampling decoding path fed into the Conditional U-Net denoising network uses this fusion feature as the basis for the backdiffusion process during the gradual denoising. This ensures that the crack structure morphology, crack direction and background texture are consistently constrained according to the text semantics in the entire space, so as to ensure that the final crack augmented image is highly consistent with the text description at the semantic level.

[0084] S150. Based on the crack area ratio of the original crack image, a crack-aware dynamic noise scheduling strategy is constructed, and the noise parameters at the corresponding time steps are adjusted according to the crack-aware dynamic noise scheduling strategy to train the conditional U-Net denoising network according to the noise parameters.

[0085] Specifically, crack area ratio is an indicator used to quantify the spatial proportion of cracks in the original crack image. It is generally statistically analyzed using crack masking or crack segmentation results, and is expressed as the ratio of the number of pixels in the crack area to the total number of effective pixels in the entire image. Crack area ratio reflects the "size" and "sparseness" of cracks in the image. When the crack area ratio is small, it usually means that there are fine or micro-cracks in the image, and the cracks only occupy a small part of the image. For example, a high-resolution image of a concrete wall may only have a hairline-sized crack. When the crack area ratio is large, it may indicate a scene with large-scale through cracks, large-area peeling, or crazing, such as a road surface with a large network of crazing, where a large area of ​​the entire image is covered by cracks. As a quantifiable indicator, crack area ratio provides an important basis for subsequent adjustments to noise intensity and training strategies.

[0086] The crack-aware dynamic noise scheduling strategy refers to a method that, when modeling the diffusion noise addition and reverse denoising of original crack images, instead of using a uniform and fixed noise scheduling curve for all images, adaptively assigns different noise evolution trajectories to different crack scenarios based on their respective crack area proportions. This ensures that images with fine cracks and micro-cracks are less damaged in the early stages of noise addition, while images with large-area cracks or coarse cracks can withstand higher noise intensities. This achieves a balance between preserving the crack structure of small targets and improving the robustness of the model. For example, for micro-crack images with extremely small crack area proportions, adding a large amount of noise in the early stages of noise addition can easily completely submerge the edges of the fine cracks within one or two steps, making reverse diffusion virtually unrecoverable. The crack-aware dynamic noise scheduling strategy sets a lower noise intensity for these images in the first few time steps and slows down noise growth, ensuring sufficient preservation of the fine crack outlines. Conversely, for cracked pavement images with large crack area proportions, a noise curve close to conventional cosine or linear scheduling can be used, allowing the network to learn to recover more obvious crack structures under high noise conditions.

[0087] In diffusion models and conditional U-Net denoising networks, a time step is used to mark the discrete stages of the noise addition and denoising process, describing the "noise evolution stage" of the current image. Typically, the entire diffusion process is discretized into several time steps, starting from an initial clean image, with noise gradually added as the time step increases until it approaches a state of near-complete noise. In the back-diffusion process, it starts from a high-noise time step and gradually works backwards to a low-noise time step until a clear image is restored.

[0088] Noise parameters are a set of control quantities describing how noise is superimposed on the image at each time step. They typically include noise variance, noise weighting coefficients, and the mixing ratio of image and noise. Noise parameters determine the degree to which the original crack image content is preserved at the current time step and the proportion of random noise. When the noise intensity corresponding to the noise parameter is low, the image still mainly presents a recognizable crack structure and background texture; when the noise intensity corresponding to the noise parameter is high, the image will gradually show random textures, and the crack structure will be obscured. Adjusting the noise parameters at corresponding time steps based on a crack-aware dynamic noise scheduling strategy means that for images with different crack area proportions, different noise intensities or different noise weights are used at the same time step, thus reflecting an adaptive characteristic of "depending on the crack" during the noise addition process. For example, a smaller noise variance is used for micro-crack images at time steps one to fifty, while a medium level of noise variance is used for large crack images at the same time step; these differences are reflected in the noise parameters.

[0089] Furthermore, firstly, the number of crack pixels is counted based on the crack mask image of the original crack image, and the ratio of the number of crack pixels to the number of effective pixels is used as the crack area ratio. Specifically, the original crack image is represented in the spatial dimension as a shape of size [missing information]. The pixel grid represents the corresponding crack mask image as a binary or multi-valued image, in pixel coordinates. The mask value at that location is denoted as The method involves using a threshold to count the pixels belonging to the crack region, and then counting all valid pixels in the entire image to obtain the crack area percentage. , can be represented as:

[0090]

[0091] in, Represents the number of pixels in the crack, by applying all the conditions... The pixels are counted to obtain the result. This represents the crack mask threshold, used to distinguish cracked areas from non-cracked areas. Its value range is usually in the middle to high range of the mask grayscale value range. This represents the number of valid pixels, obtained by counting all pixels with valid markers, such as those without invalid markers. This is used to exclude interference from unlabeled or cropped areas in an image; This indicates an indicator function that takes the value of 1 if the condition within the parentheses is true, and takes the value of 0 otherwise. This represents the percentage of the crack area, with a value between zero and one. A value close to zero indicates that the cracked area in the image is extremely small, representing a scene with fine or micro-cracks. Approximately one indicates that most of the image consists of cracked or peeling areas, such as severely cracked or extensively damaged scenes. This statistical method can provide a quantitative description of crack size for subsequent noise scheduling and training strategies without altering the image content.

[0092] After obtaining the crack area ratio, the noise intensity baseline and noise growth rate of the basic cosine noise scheduling curve are adaptively adjusted based on the crack area ratio to generate noise parameters for different time steps. First, the normalized time variable of the basic cosine scheduling is set. Then, amplitude modulation function and exponential modulation function related to the crack area ratio are constructed to simultaneously control the overall level of noise intensity and its growth rate over time, thereby obtaining adaptive noise parameters for different crack area ratios. One implementation method can be represented as:

[0093]

[0094]

[0095]

[0096] in, This represents the current time step index, which is one to... Integers between [a certain number] Represents the total number of time steps, and is an integer greater than one; This represents the time shift offset, used to avoid the curve being too steep at the initial time step. Its value is usually a small positive number between zero and several time steps. Represents the time-nonlinear compression coefficient, when The early stage time period is lengthened and the later stage time period is compressed, thus concentrating the noise growth in the middle and late stages. and These represent the lower and upper limits of the noise intensity benchmark value, respectively, used to limit the overall noise level and meet the following requirements. ; This represents the sensitivity coefficient of the crack area ratio to noise amplitude modulation. Larger changes This will cause significant amplitude changes; This represents the amplitude modulation function obtained based on the proportion of crack area. Approaching zero Approaching This is beneficial for protecting microcracks, when Approaching a time Approaching This increases noise disturbance; and These represent the lower and upper limits of the noise growth rate exponent, respectively, satisfying... ; This indicates the degree of nonlinearity of the crack area ratio in exponential modulation, when The proportion of larger crack areas has a more significant impact on the growth rate; An exponential parameter representing the change in the proportion of crack area is used to control the steepness of the transition of the cosine curve from low noise to high noise. Indicates at time step Corresponding, the proportion of crack area is The noise parameters at that time, with values ​​ranging from zero to... Between, when When smaller, Maintaining a low level in the early time step slows down the rate at which fine cracks are destroyed. When it is large, It will improve rapidly in the mid-to-late stages, enabling the network to learn the recovery ability of large crack structures under high noise conditions.

[0097] During the training of the Conditional U-Net denoising network, noise parameters are selected for each time step based on the crack area ratio. These noise parameters are then used to add noise to the original crack image, thus constructing training samples covering different crack sizes and noise intensities. This allows the network to learn denoising mappings under various conditions. For a given original crack image, its pixel value tensor is denoted as... ,in Indicates the pixel position index. Indicates the channel index; random sampling time steps are performed during training. First, let's look at the information given in the previous paragraph. Calculate the corresponding cumulative retention coefficient Combined with Gaussian noise field Generate time step Noisy images As the training input for the conditional U-Net denoising network, it can be expressed as:

[0098]

[0099] in, Indicates at time step The time, the proportion of crack area is The noise parameters are consistent with the noise scheduling function defined in the previous section; Indicates at time step The cumulative preservation coefficients of the original image components are obtained by analyzing the data from step one to step two. The noise parameters are obtained by multiplying them step by step, and their values ​​range from zero to one. Approaching a certain point indicates that the original image information still dominates, when A value close to zero indicates that noise is the dominant component; Indicates at time step Gaussian noise field at pixel position ,aisle The noise value at a given location is typically sampled from an independent Gaussian distribution with zero mean and unit variance. The noise balance coefficient, which represents the proportion of crack area, can be set to... This is used to reduce noise weights in small crack scenarios and to moderately enhance noise weights in large crack scenarios. Indicates at time step The proportion of crack area is The pixel values ​​of the noisy image generated in time are obtained by normalizing and weighting the original image and the noisy image. When training the conditional U-Net denoising network, the pixel values ​​of the noisy image are... The time step index and text semantic embedding vector are fed into the network together. The loss function is constructed based on the difference between the network's output noise prediction result or clean image reconstruction result and the real noise or real image. A large number of different crack area proportions are utilized. Different time steps and different noise fields The training samples are iteratively optimized so that the trained conditional U-Net denoising network can apply appropriate denoising intensity for fine cracks and large cracks respectively during the actual inference stage, effectively restoring crack structure details and maintaining background texture consistency.

[0100] S160. Based on the trained conditional U-Net denoising network, random noise is used as the initial image during the inference phase. The random noise and the text semantic embedding vector are input into the back diffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has the ability to adapt to realistic textures and complex backgrounds.

[0101] Specifically, an initial image is first constructed at the beginning of the inference phase. Then, at each time step, the initial image or the image generated in the previous time step, the text semantic embedding vector, and the current time step index are fed into the trained conditional U-Net denoising network to obtain the denoised feature output for the current time step. Specifically, let the total number of backdiffusion time steps be... In the initial stage of inference, the initial image tensor is obtained by sampling from a Gaussian random noise distribution. The image state at the noisiest time step can be represented as:

[0102]

[0103] in, Indicates at time step The initial image tensor, where the pixel positions and channel values ​​in the image are derived from zero mean and variance. Independent Gaussian random variables; This represents the initial noise intensity coefficient, the value of which is determined by the generation task's requirement for a balance between diversity and stability. This represents a diagonal matrix matching the image dimensions, used to indicate independent sampling of each dimension. This is then applied to the time-step sequence. Iterate over the time step, for any time step Get the current image tensor With text semantic embedding vectors and the time step embedding vector obtained through the time position encoding function They are fed together into the trained Conditional U-Net denoising network. and through the feature projection operator Obtain denoised feature output

[0104]

[0105] in, Indicates the first Time step in the next reasoning attempt The corresponding current image tensor; This represents the text semantic embedding vector obtained from the text describing the crack attributes, which includes information such as crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics, and crack material semantics. Indicates the index of discrete time steps The time step embedding vector obtained by mapping sine and cosine position encoding with a multilayer perceptron is used to explicitly provide the network with the current stage position in the back diffusion process; This represents a conditional U-Net denoising network that has converged and had its parameters fixed during the training phase. The parameter set is denoted as... ; This represents a projection or normalization operator used to extract multidimensional features from the network output that are related to noise estimation and explicit structure reconstruction. It typically consists of a set of convolutional layers, normalization layers, and nonlinear activation layers. Indicates at time step The resulting denoised feature output contains both an estimate of the current noise distribution and information on the tendency to reconstruct the crack structure and background texture.

[0106] After obtaining the denoised feature output at the current time step, the image for the next time step is reconstructed based on this denoised feature output, the text semantic embedding vector, and the corresponding noise parameters. This allows the back-diffusion process to progressively reduce noise intensity and enhance crack structure, crack texture, and background texture information across multiple time steps. After completing the image reconstruction iteration at each time step, a crack augmentation image is output when the time step degenerates to its final state. This ensures that the crack augmentation image maintains structural and semantic consistency with the crack attribute description text, forming a high-quality sample that can be directly used for data augmentation. Specifically, when the time step... Updated to the first time, the resulting image tensor Post-processing operator Mapping to the target image value range, the final crack augmentation image is obtained as follows:

[0107]

[0108] in, Indicates the first The crack augmentation image generated during the next inference process; The post-processing operators can include operations such as pixel value cropping, normalization to the display range, color space conversion, and gamma correction to ensure that the generated image meets the requirements of display and downstream tasks. Since each time step update in the entire back-diffusion process simultaneously relies on the denoised feature output and the text semantic embedding vector, and the noise scheduling parameters are adaptively adjusted according to the crack area ratio, the generated crack augmented image will present crack direction, crack width, and crack shape consistent with the crack attribute description text at a macroscopic level, and crack texture and background texture details matching the target material at a local level. This satisfies the requirement of "generating the crack that describes it" in terms of structural consistency, and the requirement of "matching the generated result with the described semantics in scale, direction, material, and complexity" in terms of semantic consistency. This provides high-quality augmented samples with realistic texture and complex background adaptation capabilities for downstream tasks such as crack detection and crack segmentation.

[0109] This application also provides a crack data augmentation device, referring to... Figure 2 , Figure 2 This is a schematic diagram of a crack data augmentation device provided in an embodiment of this application. The device is a server, which includes an acquisition module 21 and a processing module 22. The acquisition module 21 is used to acquire crack attribute description text and original crack images, and generate text semantic embedding vectors by semantic parsing and sequence encoding of the crack attribute description text. The processing module 22 is used to perform multi-stage noise addition processing on the original crack images based on cosine noise scheduling to generate noisy image sequences covering different noise intensities. The processing module 22 is also used to input the noisy image sequences, text semantic embedding vectors and corresponding time step indices into a conditional U-Net denoising network, and extract multi-layer semantic features of the images through a multi-scale convolutional structure. The multi-layer semantic features of the images are used to characterize crack structure features and background texture features. The processing module 22 is also used to construct text in the bottleneck layer of the conditional U-Net denoising network. - The image cross-attention structure fuses text semantic embedding vectors and multi-layer semantic features of images in both spatial and semantic dimensions through the text-image cross-attention structure, making the back-diffusion process subject to text semantic constraints; Processing module 22 is also used to construct a crack-aware dynamic noise scheduling strategy based on the crack area ratio of the original crack image, and adjust the noise parameters at the corresponding time step according to the crack-aware dynamic noise scheduling strategy, so as to train the conditional U-Net denoising network according to the noise parameters; Processing module 22 is also used to use random noise as the initial image in the inference stage of the trained conditional U-Net denoising network, and input the random noise and text semantic embedding vector together into the back-diffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has the ability to adapt to realistic textures and complex backgrounds.

[0110] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0111] This application also provides an electronic device, with reference to... Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 31, at least one network interface 34, a user interface 33, a memory 35, and at least one communication bus 32.

[0112] The communication bus 32 is used to enable communication between these components.

[0113] The user interface 33 may include a display screen and a camera. Optionally, the user interface 33 may also include a standard wired interface and a wireless interface.

[0114] The network interface 34 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0115] The processor 31 may include one or more processing cores. The processor 31 connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in the memory 35, and calling data stored in the memory 35 to perform various server functions and process data. Optionally, the processor 31 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 31 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 31 and may be implemented as a separate chip.

[0116] The memory 35 may include random access memory (RAM) or read-only memory. Optionally, the memory 35 may include a non-transitory computer-readable storage medium. The memory 35 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 35 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 35 may also be at least one storage device located remotely from the aforementioned processor 31. Figure 3 As shown, the memory 35, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a crack data augmentation method.

[0117] exist Figure 3 In the electronic device shown, the user interface 33 is mainly used to provide an interface for users to input data and obtain user input data; while the processor 31 can be used to call an application program stored in the memory 35 for a crack data augmentation method. When executed by one or more processors, the electronic device performs one or more methods as described in the above embodiments.

[0118] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0119] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause an electronic device to perform one or more of the methods described in the above embodiments.

[0120] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0125] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for augmenting crack data, characterized in that, The method includes: Obtain crack attribute description text and original crack image, and generate text semantic embedding vector by semantic parsing and sequence encoding of crack attribute description text; The original crack image is subjected to multi-stage noise addition processing based on cosine noise scheduling to generate a noisy image sequence covering different noise intensities; The noisy image sequence, the text semantic embedding vector, and the corresponding time step index are input into a conditional U-Net denoising network, and multi-scale semantic features of the image are extracted through a multi-scale convolutional structure. The multi-scale semantic features of the image are used to characterize crack structure features and background texture features. A text-image cross-attention structure is constructed in the bottleneck layer of the conditional U-Net denoising network. The text-image cross-attention structure is used to fuse the text semantic embedding vector and the multi-layer semantic features of the image in both spatial and semantic dimensions, so that the back diffusion process is constrained by the text semantics. Based on the crack area ratio of the original crack image, a crack-aware dynamic noise scheduling strategy is constructed, and the noise parameters at the corresponding time steps are adjusted according to the crack-aware dynamic noise scheduling strategy, so as to train the conditional U-Net denoising network according to the noise parameters. Based on the trained Conditional U-Net denoising network, random noise is used as the initial image during the inference phase. The random noise and the text semantic embedding vector are input into the back diffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has the ability to adapt to realistic textures and complex backgrounds. The process of constructing a crack-aware dynamic noise scheduling strategy based on the crack area ratio of the original crack image, and adjusting the noise parameters at corresponding time steps according to the crack-aware dynamic noise scheduling strategy, in order to train the conditional U-Net denoising network based on the noise parameters, specifically includes: The number of crack pixels is counted based on the crack mask image of the original crack image, and the ratio of the number of crack pixels to the number of effective pixels is used as the crack area ratio. The noise parameters are generated by adaptively adjusting the noise intensity benchmark value and noise growth rate of the basic cosine noise scheduling curve based on the crack area ratio. During the training process of the conditional U-Net denoising network, the corresponding noise parameters are selected for each time step based on the crack area ratio, and the original crack image is subjected to noise addition processing based on the noise parameters to obtain the trained conditional U-Net denoising network.

2. The crack data augmentation method according to claim 1, characterized in that, The process of obtaining crack attribute description text and original crack image, and generating text semantic embedding vector by semantic parsing and sequence encoding of the crack attribute description text, specifically includes: The crack attribute description text is processed by text regularization, word boundary parsing and semantic tagging to form a structured text sequence; The structured text sequence is input into the text encoding model, and a word vector sequence is generated through the word vector embedding layer of the text encoding model; The word vector sequence is input into the semantic parsing network structure, and the contextual associations between crack width semantics, crack direction semantics, crack orientation semantics, crack morphology semantics and crack material semantics are captured through a multi-layer self-attention mechanism. A global sequence compression operation is performed on the context association in the sequence coding network to generate the text semantic embedding vector.

3. The crack data augmentation method according to claim 1, characterized in that, The multi-stage noise addition process based on cosine noise scheduling is applied to the original crack image to generate a noisy image sequence covering different noise intensities, specifically including: A continuous time step sequence is constructed based on the cosine noise scheduling, and a noise intensity ratio determined by the cosine variation trend is assigned to each time step. In each time step, a pixel-level noise mixing operation is performed on the original crack image according to the corresponding noise intensity ratio, so that the content of the original crack image is proportionally fused with the random noise field to generate a noisy image. The noisy images generated at each time step are sequentially stored into a multi-level sequence to form the noisy image sequence.

4. The crack data augmentation method according to claim 1, characterized in that, The step of inputting the noisy image sequence, the text semantic embedding vector, and the corresponding time step index into a conditional U-Net denoising network, and extracting multi-layer semantic features of the image through a multi-scale convolutional structure, specifically includes: The noisy image sequence is mapped to a base feature map, and the time step index is encoded into a time step feature vector. The time step feature vector is then channel-fused with the base feature map through feature broadcasting to obtain a time step fused feature map. The text semantic embedding vector is mapped to a semantic feature vector, and the semantic feature vector is concatenated or additively fused with the basic feature map to form a semantic fusion feature map. The time-step fusion feature map and the semantic fusion feature map are sequentially input into the multi-level downsampling convolutional block of the conditional U-Net denoising network to extract low-level semantic information and high-level semantic information. The spatial resolution is restored by multi-level upsampling convolutional blocks of the conditional U-Net denoising network, and cross-layer feature concatenation is performed. The low-level semantic information and the high-level semantic information are combined to generate the multi-layer semantic features of the image.

5. The crack data augmentation method according to claim 1, characterized in that, The text-image cross-attention structure is constructed in the bottleneck layer of the conditional U-Net denoising network. This structure fuses the text semantic embedding vector with the multi-layer semantic features of the image in both spatial and semantic dimensions, thus constraining the back-diffusion process by text semantics. Specifically, this includes: The multi-layer semantic features of the image are expanded into image sequence features, and the text semantic embedding vector is mapped into text sequence features, so that the image sequence features and the text sequence features maintain a fusionable relationship in the channel dimension; The image sequence features are used as query vectors, and the text sequence features are used as key vectors. The text-image cross-attention structure is then used to generate semantic response features by calculating the semantic association weights between the query vectors and the key vectors. The semantic response features are reconstructed into a spatial feature map and fused with the multi-layer semantic feature execution channel of the image, so that the back diffusion process is constrained by the text semantics.

6. The crack data augmentation method according to claim 1, characterized in that, The conditional U-Net denoising network, trained according to the prior art, uses random noise as the initial image during the inference phase. The random noise and the text semantic embedding vector are then input into the back-diffusion process to generate a crack augmented image that corresponds to the crack attribute description text and possesses realistic texture and adaptability to complex backgrounds. Specifically, this includes: The initial image is initialized at the start of inference. At each time step, the initial image or the image generated at the previous time step, the text semantic embedding vector, and the index of the current time step are input into the trained conditional U-Net denoising network to generate the denoising feature output of the current time step. Based on the denoising feature output, the text semantic embedding vector and the corresponding noise parameters, the image of the next time step is reconstructed, so that the back diffusion process gradually reduces the noise intensity and gradually strengthens the crack structure information, crack texture information and background texture information in multiple time steps. After completing the reconstruction at each time step, the crack augmentation image is output, ensuring that the crack augmentation image and the crack attribute description text maintain structural and semantic consistency.

7. A crack data augmentation device, characterized in that, The apparatus is used to perform the crack data augmentation method as described in any one of claims 1 to 6, the apparatus comprising an acquisition module and a processing module, wherein... The acquisition module is used to acquire crack attribute description text and original crack image, and generate text semantic embedding vector by semantic parsing and sequence encoding of crack attribute description text. The processing module is used to perform multi-stage noise addition processing on the original crack image based on cosine noise scheduling to generate a noisy image sequence covering different noise intensities. The processing module is further configured to input the noisy image sequence, the text semantic embedding vector and the corresponding time step index into the conditional U-Net denoising network, and extract multi-layer semantic features of the image through a multi-scale convolutional structure. The multi-layer semantic features of the image are used to characterize crack structure features and background texture features. The processing module is also used to construct a text-image cross-attention structure in the bottleneck layer of the conditional U-Net denoising network, and to fuse the text semantic embedding vector and the multi-layer semantic features of the image in the spatial and semantic dimensions through the text-image cross-attention structure, so that the back diffusion process is constrained by the text semantics. The processing module is further configured to construct a crack-aware dynamic noise scheduling strategy based on the crack area ratio of the original crack image, and adjust the noise parameters at the corresponding time step according to the crack-aware dynamic noise scheduling strategy, so as to train the conditional U-Net denoising network according to the noise parameters. The processing module is further configured to use random noise as the initial image during the inference phase of the trained Conditional U-Net denoising network, and input the random noise and the text semantic embedding vector into the back diffusion process to generate a crack augmented image that corresponds to the crack attribute description text and has the ability to adapt to realistic textures and complex backgrounds.

8. An electronic device, characterized in that, The electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 6.

Images