Structure apparent anomaly cross-modal identification method and system based on diffusion model

Abstract

The invention provides a structure apparent anomaly cross-modal identification method and system based on a diffusion model, and relates to the technical field of anomaly identification, and the method comprises the steps: obtaining a to-be-identified apparent image of a target structure body; constructing an exception recognition instruction for the exception type to be recognized; inputting a to-be-recognized apparent image and an anomaly recognition instruction into the trained image-text cross-modal recognition model, executing a visual task of anomaly recognition, and obtaining a target image containing a semantic segmentation result; wherein the image-text cross-modal recognition model is constructed based on a diffusion model, and a vision task of abnormal recognition is converted into image generation constrained by an image condition and an instruction condition; according to the method, the apparent images and task instructions of the engineering and geologic structures are input into the constructed image-text cross-modal recognition model, the semantic segmentation result of the images is obtained, and apparent anomaly pixel-level refined recognition is achieved.

Description

Technical Field

[0001] This invention relates to the field of anomaly recognition technology, specifically to a method and system for cross-modal recognition of structural appearance anomalies based on a diffusion model. Background Technology

[0002] Engineering and geological structures (including infrastructure projects and rock and soil geological bodies) are continuously affected by environmental factors and loads during long-term service or natural evolution, leading to abnormal modes. For engineering structures, this manifests as material deterioration such as concrete cracks, steel corrosion, and joint leakage; for geological structures, it may manifest as geological defects such as rock mass fissure expansion, soil slippage, and dissolution cavities. If these anomalies are not identified and intervened in a timely manner in their early stages, local damage often spreads gradually through a chain reaction—engineering structures may experience load-bearing capacity degradation or sudden collapse, while geological structures are prone to chain disasters such as landslides and ground subsidence. This evolutionary process from microscopic damage to macroscopic failure ultimately poses a significant threat to the safety of human life and property and the ecological environment.

[0003] With the rapid development of artificial intelligence and digital image processing technologies, intelligent detection based on deep learning has become an important trend in the intelligent and automated development of structural anomaly detection, replacing manual visual inspection. Currently, mainstream deep learning-based anomaly detection methods are mostly based on deep convolutional neural networks and encoder-decoder structures such as U-Net, DeepLabV3+, and SegFormer. Deep learning methods exhibit strong advantages over traditional methods, but their development is still constrained by two core bottlenecks: first, insufficient model generalization ability, leading to a significant drop in detection performance when facing complex and ever-changing real-world scenarios; second, over-reliance on large amounts of finely labeled data, resulting in high model costs and difficulty in quickly adapting to new scenarios.

[0004] Therefore, existing structural appearance anomaly identification technologies are insufficient in terms of high accuracy and strong generalization ability. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a cross-modal recognition method and system for structural appearance anomalies based on a diffusion model. The method inputs appearance images of engineering and geological structures and task instructions into a constructed image-text cross-modal recognition model to obtain semantic segmentation results of the images, thereby achieving pixel-level refined recognition of appearance anomalies.

[0006] According to some embodiments, the present invention adopts the following technical solution: A cross-modal identification method for structural appearance anomalies based on a diffusion model includes: Obtain the appearance image of the target structure to be identified; For each type of anomaly to be identified, construct anomaly identification instructions; The appearance image to be identified and the anomaly identification instructions are input into the trained image-text cross-modal recognition model to perform the visual task of anomaly identification and obtain the target image containing semantic segmentation results. The image-text cross-modal recognition model is constructed based on a diffusion model, which transforms the visual task of anomaly recognition into image generation constrained by image conditions and instruction conditions.

[0007] According to some embodiments, the present invention adopts the following technical solution: A cross-modal identification system for structural appearance anomalies based on a diffusion model, comprising: The image acquisition module is configured to acquire the appearance image of the target structure to be identified. The instruction building module is configured to: build exception recognition instructions for the exception type to be identified; The anomaly detection module is configured to: input the appearance image to be identified and the anomaly detection instruction into the trained image-text cross-modal recognition model, perform the visual task of anomaly detection, and obtain the target image containing the semantic segmentation result; The image-text cross-modal recognition model is constructed based on a diffusion model, which transforms the visual task of anomaly recognition into image generation constrained by image conditions and instruction conditions.

[0008] According to some embodiments, the present invention adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the aforementioned method for cross-modal identification of structural appearance anomalies based on a diffusion model.

[0009] According to some embodiments, the present invention adopts the following technical solution: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned method for cross-modal identification of structural appearance anomalies based on a diffusion model.

[0010] According to some embodiments, the present invention adopts the following technical solution: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned cross-modal identification method for structural appearance anomalies based on a diffusion model.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention introduces the powerful generative prior of the diffusion model, enabling the model to deeply understand the general visual features and semantic concepts of various anomalies. In particular, the introduction of image guidance coefficients and instruction guidance coefficients ensures a balance between semantic parsing and image details, achieving robust and accurate identification of various structural anomalies (such as cracks, peeling, corrosion, etc.), greatly improving the generalization and adaptability of the method in complex real-world environments. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0013] Figure 1 This is a flowchart of the method in Example 1. Detailed Implementation The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0014] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0015] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0016] Example 1 One embodiment of the present invention provides a method for cross-modal identification of structural appearance anomalies based on a diffusion model, such as... Figure 1 As shown, it includes: Step S1: Obtain the appearance image of the target structure to be identified; Step S2: Construct anomaly recognition instructions for the anomaly type to be identified; Step S3: Input the appearance image to be identified and the anomaly identification instruction into the trained image-text cross-modal recognition model, perform the visual task of anomaly identification, and obtain the target image containing the semantic segmentation result; The image-text cross-modal recognition model is constructed based on a diffusion model, which transforms the visual task of anomaly recognition into image generation constrained by image conditions and instruction conditions.

[0017] In recent years, diffusion models, as an emerging generative model, have gradually attracted widespread attention in academia due to their superior performance in image generation tasks. By simulating the progressive noise process of images, diffusion models can recover the original data from the noise, generate higher-fidelity images, and have more flexible model architectures and accurate log-likelihood calculations. At the same time, the rapid development of natural language processing technology, especially instruction learning and cross-modal modeling technologies such as CLIP, has made semantic bridging between images and text possible, providing a technical foundation for "language-guided image editing".

[0018] Based on the aforementioned technological trends, this embodiment proposes a cross-modal recognition method for structural appearance anomalies based on a diffusion model. Cross-modality refers to the process of information transfer and understanding between different modalities (such as visual and textual). The InstructDiffusion cross-modal recognition model, constructed based on a diffusion model, is introduced into the field of infrastructure appearance anomaly detection. By constructing a closed-loop framework of "natural language instructions - cross-modal inference - pixel-level segmentation," unified recognition and accurate localization of multiple types of structural anomalies under zero-shot conditions are achieved. This embodiment effectively compensates for the shortcomings of existing methods in terms of generalization and interactivity, providing a more versatile new solution for intelligent infrastructure structure inspection systems. The specific implementation process is described in detail below from the perspective of the construction and training of the InstructDiffusion recognition model: I. Data Acquisition and Preprocessing Images of apparent anomalies in engineering or geological structures are acquired using devices such as mobile phones, cameras, or drones. The acquired images undergo uniform image size normalization and illumination correction to reduce interference from the external environment on the model's recognition accuracy. Simultaneously, image enhancement techniques such as edge enhancement and contrast enhancement are used to highlight features in target areas such as cracks and corrosion, facilitating accurate identification by the subsequent InstructDiffusion recognition model.

[0019] II. InstructDiffusion, a cross-modal image-text recognition model Stable Diffusion is an efficient text-guided image generation model based on the Denoising Diffusion Probabilistic Model (DDPM). DDPM includes forward diffusion and backward diffusion processes. First, random noise is gradually added to the sample data during forward diffusion, and then samples are generated from the noise through backward diffusion. During forward diffusion, it is assumed that... Given the original data distribution and undisturbed training samples... The scrambled data sequence is obtained through the following Markov process. : (1) in, For sample-based Noise is added to the samples according to a Gaussian distribution to generate the next sample. , t Indicates the diffusion step. T The number of diffusion steps; Indicates the corresponding diffusion step as t The variance of the noise added at each step is used to control the noise intensity added at each step; Indicates and Identity matrices with the same dimensions; express x Obtained from the mean Covariance is It follows a normal distribution.

[0020] Any sample It can be calculated from equation (2), that is, only an initial sample is required. and variance This allows for the completion of arbitrary diffusion steps to disrupt data in one step. Sampling: (2) in, .

[0021] Using parameter renormalization, one can... Samples obtained from sampling : (3) From equation (3), we can see that if the variance , The distribution of can approximate the standard normal distribution quite well. .if If the value is much less than 1, then the reverse process applies. With forward process They have the same function form.

[0022] Utilizing the properties of the diffusion process, if from random Gaussian noise Start sampling, and the reverse process follows... You can get A new sample. However, because it cannot be obtained from... get Therefore, it is necessary to train a parameterized deep neural network. To approximate The input to this network is a scrambled image x. t and diffusion step t Learning and predicting the mean Covariance The denoised image x is obtained. t-1 .

[0023] The neural network is trained using maximum likelihood estimation, which enables it to predict the original image. Deep neural networks Assigned to each training sample The probability should be as high as possible, but It is difficult to calculate and needs to be done by minimizing the variational lower bound of the negative log-likelihood: (4) in, This represents the Kullback-Leibler divergence between two probability distributions.

[0024] Due to variance It has been fixed as a constant between 0 and 1, so L vlb The first item It is a constant; the second term This indicates that when the original image is introduced... When given known conditions, training the neural network makes each diffusion step... All of these can make As close as possible to the true posterior distribution of the forward diffusion process Due to parameterized deep neural networks To approximate The equation before parameter renormalization can be expressed as: And because of variance Set as a with Related constants and They are equal, expressed by the formula: (5) Therefore, trainable parameters only exist in the mean. In, and satisfy: (6) in, Indicates prediction Neural networks with moderate noise It is obtained by sampling through equation (3).

[0025] The objective function can be obtained from the above simplification. and reverse diffusion process The expressions (7) and (8), i.e., the noise prediction network The training objective is to minimize the prediction noise. and real noise The differences between them.

[0026] (7) (8) In Stable Diffusion, both diffusion and anti-diffusion occur in a low-dimensional latent space. Therefore, firstly, the encoder is generated by a pre-trained Variational Auto-Encoder (VAE). E (·) Mapping image data to latent variables z 0. This latent variable is usually a compressed representation of the original image space, used to carry the structural and texture information of the image. Then, the forward diffusion process gradually injects Gaussian noise into the latent variable, causing it to gradually degenerate from the real sample to a pure noise distribution. In the reverse diffusion process, the model learns a denoising network U-Net to gradually predict and remove noise, thereby realizing the recovery from random noise to structured latent variables.

[0027] The image-text cross-modal recognition model InstructDiffusion in this embodiment is built based on the diffusion model StableDiffusion. It transforms the visual task of anomaly recognition into image generation constrained by image conditions and instruction conditions, wherein the instruction conditions are provided by a language model. The task instructions are encoded to obtain a semantic embedding vector, which captures the semantic information of the text and provides semantic constraints for image generation. In the de-diffusion stage, the latent variables of diffusion step T... z T Feature updates are performed using a denoised U-Net. The semantic embedding vector of the instruction is embedded into the multi-layer feature space of the U-Net through cross-attention, and its calculation form is as follows:

[0028] In this model, the current layer image features of U-Net are represented by Q, which is the feature query vector of the image region to be "guided". Semantic embedding vectors are represented by K and V, where K represents the feature "index" of each semantic embedding vector, and V represents the feature "content value" of the corresponding semantic embedding vector. This allows the model to learn to focus on semantically relevant regions in the latent space, thereby achieving language-guided image generation and editing.

[0029] When the reverse diffusion process is complete T After a step of reverse reasoning, the final denoised latent variables are obtained. via VAE decoder D (·) Map back to the image space to obtain the target image, thereby realizing cross-modal generation from text to image and visualization of structural anomalies.

[0030] The training objectives of the image-text cross-modal recognition model are as follows: (10) in, It is the semantic embedding vector of the task instruction.

[0031] During the inference phase, from standard Gaussian noise Initially, based on instruction-conditional embedding, a learned denoising network is used. Execute sequentially T The second reverse diffusion step ultimately passes through the decoder. D (·) Obtain the original image after denoising.

[0032] InstructDiffusion reformulates various computer vision tasks as conditional image-to-image generation problems constrained by natural language instructions. Based on Stable Diffusion, the training dataset consists of triplets (source image, target image, task instruction), where the task instruction precisely describes the semantic transformation relationship between the source and target images in natural language. The visual model Stable Diffusion is responsible for performing semantic-level transformations on the images according to these language instructions, generating the corresponding target images. This forms a mapping pair of "text semantics—image transformation," ensuring that each training sample contains semantically interpretable task relationships. The generated dataset is used to train a conditional diffusion network to predict noise in the target latent variable. Classifier-Free Guidance (CFG) technology is incorporated to improve the semantic consistency between the generated output and the text editing instructions by introducing an image guidance coefficient. s I With text guidance coefficient s T This allows for fine-grained control over the editing intensity between image content and language commands. Specifically, it means: (11) in, The final noise predicted after comprehensive weighting, This represents the visual conditions obtained by encoding the input image using VAE. A semantic embedding vector representing a task instruction. This represents the latent representation of a noisy image during the diffusion process. This indicates that the corresponding input is set to empty, thus obtaining an unconditional branch. For noise predicted without conditions, For noise predicted solely by image conditions, This refers to the noise predicted under dual conditions of image and instruction.

[0033] III. Model Optimization 1. CFG hyperparameter tuning The CFG hyperparameter adjustment step employs an adaptive optimization algorithm, specifically a gradient descent method based on dynamic weight adjustment. This method uses the feature separation index as the optimization objective function, calculates the Euclidean distance between the anomaly region and the background region in the embedding space, and uses this separation as a feedback signal input to the optimizer.

[0034] The feature separation index measures the degree to which outlier regions and background regions are distinguishable in the model embedding space. Let the feature mean of the outlier region be... The feature mean of the background region is The feature separation degree is defined as: (12) in, This represents the Euclidean distance. The optimizer uses this signal to guide the text coefficients s. T With image guidance coefficient s I The weights are iteratively updated. The Adam optimization algorithm is used to constrain the weight update process to avoid oscillations and overfitting.

[0035] Furthermore, in When the speed decreases, the optimizer increases. s T The weight; in When the speed increases, the optimizer should be appropriately increased. s I The weights are determined to ensure a balance between semantic parsing and image details.

[0036] Preferably, the text guidance coefficient s T Set to 5.0 to enhance the model's response to semantic instructions; the image guidance coefficient s I Set to 1.25 to preserve detail features in the image.

[0037] 2. Optimization of inference efficiency The inference efficiency optimization employs a combination of early stopping convergence strategy and Pareto optimal solution set constraints to achieve a dynamic balance between accuracy and efficiency. During inference, intermediate segmentation results are first generated quickly using a lower sampling number, and multi-objective performance metrics such as mean pixel accuracy (mPA), Dice coefficient, and IoU (Intersectionover Union) are calculated. The system constructs a performance evaluation vector using a multi-objective optimization approach. (13) in, This represents the inference time corresponding to the t-th sampling. Performance changes between consecutive sampling steps are monitored in real time, and the Pareto optimality condition is used as the criterion for early stopping. The model is considered to have reached the Pareto convergence boundary when the following equation is satisfied: (14) in, A performance change threshold (preferably set to 0.001–0.005) is used to determine whether the performance improvement is approaching saturation. When the above condition is met, the system triggers an early stop mechanism to terminate the sampling process in advance, thereby significantly shortening the inference time while ensuring recognition accuracy, achieving the optimal trade-off between accuracy and efficiency.

[0038] Furthermore, to avoid segmentation performance degradation due to insufficient sampling steps, the system introduces a dynamic step-size control mechanism near the Pareto front region. A larger learning rate is used in the early stages of convergence to quickly approximate the optimal solution set, while the learning rate is gradually reduced as the Pareto boundary approaches, allowing the performance curve to smoothly converge to a stable region. Preferably, the sampling number is 50 steps to ensure that high structural consistency and segmentation accuracy are maintained while minimizing inference time.

[0039] IV. Task Instruction Construction The prompting framework includes a task instruction construction step, employing a two-layer structure of embedded main instructions and prior knowledge. The main instructions clarify the recognition or segmentation task to be performed by the model, while the prior knowledge supplements morphological, textural, and obfuscated feature information related to structural anomalies, thereby improving the model's structural understanding ability and anomaly detection robustness.

[0040] The main command gives the task objective in natural language, with the template "Mark the pixels of %object% in red, leaving the rest unchanged.", where %object% is the anomaly category to be identified, such as "crack regions", "corrosion spots", "spalling regions", or "exposed rebars".

[0041] Both the main instructions and prior knowledge are transformed into semantic vectors by the text encoder of the language model, with the main instructions being encoded to generate the task semantic vector. C T Weak prior semantic vectors are generated after encoding prior knowledge. E prior .

[0042] Unlike traditional CFG, which uses unconditional branches that simultaneously set both text and image conditions to null, this approach is different. This embodiment employs a weak prior injection strategy, introducing weak prior semantic vectors only at text empty condition locations, rewriting the branch as follows: This allows the model to retain a low-intensity perception of anomalous morphologies during back-diffusion. This vector is introduced only in the completely unconditional branch to provide semantic background constraints, while other branches with image or text conditions retain their original structure, avoiding replacement or interference with the semantic information of the main task instructions. Therefore, the final predicted noise is expressed by the formula: (15) in, The final noise predicted after comprehensive weighting, This represents the visual conditions obtained by encoding the input image using VAE. This indicates that the task semantic vector is generated after the main instruction is encoded. This represents the latent representation of a noisy image during the diffusion process. This indicates that the corresponding input is set to empty, thus obtaining an unconditional branch. This refers to noise predicted under conditions guided only by weak prior semantic vectors. For noise predicted solely by image conditions, This refers to noise predicted under dual conditions of image and text.

[0043] This weak prior injection strategy reduces the semantic difference distance between unconditional and conditional branches without changing the original CFG gradient algebra structure, enabling inverse diffusion denoising sampling to achieve more stable anomaly region focusing capabilities.

[0044] Taking crack recognition as an example, the main instruction prompt is first constructed, such as "Mark the pixels of crackregions in red, leaving the rest unchanged.", and then extracted into a task semantic vector by a text encoder. This is used to clarify the model's output target and guide U-Net to explicitly label crack regions during the back-diffusion process. Furthermore, weak semantic text prompts are constructed using prior knowledge, such as "Cracks are thin and irregular in shape with distinct contrast from background, please avoid confusing with other lender structures.", which are extracted into weak prior semantic vectors by the text encoder. And inject into the unconditional branch. The text input position, which is about to Replace with This is to give the model a low-intensity semantic perception of crack morphology and texture.

[0045] Both the main instruction and prior knowledge participate in information fusion in the cross-attention module of U-Net. The task semantics determine the editing direction, and the prior semantics provide abnormal morphological background constraints, thereby achieving the coordinated guidance of visual features and semantic knowledge.

[0046] V. Anomaly Detection Results and Quantitative Assessment During anomaly detection, the preprocessed appearance image to be identified and the constructed task instructions are input into the InstructDiffusion model, which then outputs a segmented structural anomaly image.

[0047] The output results are binarized to visually identify the location and shape of the anomaly. Furthermore, by calculating quantitative parameters such as the length, width, and area of ​​the anomaly region, the system expands its functionality from qualitative identification to quantitative assessment, providing data support for structural health diagnosis and subsequent maintenance.

[0048] Example 2 One embodiment of the present invention provides a cross-modal identification system for structural appearance anomalies based on a diffusion model, comprising: The image acquisition module is configured to acquire the appearance image of the target structure to be identified. The instruction building module is configured to: build exception recognition instructions for the exception type to be identified; The anomaly detection module is configured to: input the appearance image to be identified and the anomaly detection instruction into the trained image-text cross-modal recognition model, perform the visual task of anomaly detection, and obtain the target image containing the semantic segmentation result; The image-text cross-modal recognition model is constructed based on a diffusion model, which transforms the visual task of anomaly recognition into image generation constrained by image conditions and instruction conditions.

[0049] Example 3 One embodiment of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for cross-modal identification of structural appearance anomalies based on a diffusion model.

[0050] Example 4 In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, they implement the aforementioned method for cross-modal identification of structural appearance anomalies based on a diffusion model.

[0051] Example 5 One embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned cross-modal identification method for structural appearance anomalies based on a diffusion model.

[0052] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for structural appearance anomaly cross-modal identification based on diffusion model, characterized in that, The method comprises: acquiring a to-be-identified appearance image of a target structure; constructing an anomaly identification instruction for a to-be-identified anomaly type; inputting the to-be-identified appearance image and the anomaly identification instruction into a trained image-text cross-modal recognition model to perform an anomaly identification visual task and obtain a target image containing a semantic segmentation result; wherein the image-text cross-modal recognition model is constructed based on a diffusion model, and the anomaly identification visual task is converted into image generation constrained by an image condition and an instruction condition.

2. The structural apparent anomaly cross-modality identification method based on a diffusion model according to claim 1, wherein, The anomaly identification instruction is composed of a main instruction and prior knowledge, wherein the main instruction is embedding the to-be-identified anomaly type into a main instruction template, and the prior knowledge includes the geometric morphology, texture features, color contrast and common confusing factors of the to-be-identified anomaly type.

3. The structural apparent anomaly cross-modality identification method based on a diffusion model according to claim 1, wherein, The image-text cross-modal recognition model adopts an efficient text-guided image generation model Stable Diffusion based on a diffusion probability model, which includes a forward diffusion process and a reverse diffusion process. First, random noise is gradually added to sample data in the forward diffusion process, and then sample data is generated from noise in the reverse diffusion process.

4. The structural apparent anomaly cross-modality identification method based on a diffusion model according to claim 3, characterized in that, The sample data generated from noise in the reverse diffusion process is predicted by a trained denoising network for each diffusion step, and the predicted noise is used for denoising.

5. The structural apparent anomaly cross-modality identification method based on a diffusion model according to claim 4, characterized in that, The denoising network adopts a classifier-free guidance technology, and image guidance coefficients and instruction guidance coefficients are introduced into the existing denoising network. The instruction guidance coefficients are used to enhance the response of the denoising network to semantic instructions, and the image guidance coefficients are used to maintain the details in the image.

6. The structural apparent anomaly cross-modality identification method based on a diffusion model according to claim 5, wherein, Further comprising: using an adaptive optimization algorithm to maximize the feature separation degree as the optimization target, iteratively updating the image guidance coefficients and the instruction guidance coefficients to balance the semantic analysis and image details.

7. A diffusion model based structural appearance anomaly cross-modal identification system, characterized in that, The method comprises: an image acquisition module configured to acquire a to-be-identified appearance image of a target structure; an instruction construction module configured to construct an anomaly identification instruction for a to-be-identified anomaly type; an anomaly identification module configured to input the to-be-identified appearance image and the anomaly identification instruction into a trained image-text cross-modal recognition model to perform an anomaly identification visual task and obtain a target image containing a semantic segmentation result; wherein the image-text cross-modal recognition model is constructed based on a diffusion model, and the anomaly identification visual task is converted into image generation constrained by an image condition and an instruction condition.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the structure appearance anomaly cross-modal recognition method based on the diffusion model according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium is used to store computer instructions, which are executed by the processor to implement the structure appearance anomaly cross-modal recognition method based on the diffusion model according to any one of claims 1-6.

10. An electronic device, comprising: The method comprises: a processor, a memory and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory to make the electronic device implement the structure appearance anomaly cross-modal recognition method based on the diffusion model according to any one of claims 1-6.

Images