Methods, systems, equipment and media for generating images of urban safety anomalies

Abstract

This invention discloses a method, system, device, and medium for generating images of urban safety anomalies, belonging to the field of computer vision and smart city technology. The technical problem this invention aims to solve is how to improve the quality and diversity of generated images of urban safety anomalies. The technical solution is as follows: Constructing and preprocessing an urban safety anomaly dataset: collecting surveillance images containing normal scenes and various anomalies; constructing a labeled map by annotating the location and category of abnormal areas in the surveillance images; dividing the dataset into training and testing sets according to a preset ratio; performing heterogeneous feature fusion to generate comprehensive conditional features: encoding the original surveillance images into latent features; downsampling the labeled map; and concatenating and fusing it with Gaussian noise features in the channel dimension; adapting the pre-trained model to the domain based on an intelligent fine-tuning framework; integrating multi-scale and spatial attention mechanisms to enhance feature extraction; performing conditionally guided image generation; and using a systematic evaluation system to quantify generation performance.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and smart city technology, specifically to a method, system, device, and medium for generating images of urban safety anomalies. Background Technology

[0002] Urban safety is a core component of smart city construction, and video surveillance systems play a crucial role in it. Deep learning-based anomaly detection methods can automatically identify safety hazards such as traffic accidents, fires, and crowd gatherings, but their performance is highly dependent on the quantity and quality of the training samples.

[0003] In real-world urban surveillance scenarios, anomalous events are characterized by low frequency, varied forms, and complex scenes, resulting in anomaly samples being extremely scarce. Traditional data augmentation methods, such as rotation, scaling, and color transformation, can only generate a limited number of sample variations and cannot generate images with novel anomalous features. In recent years, generative AI technologies, especially diffusion models, have made significant progress in image generation tasks. However, general diffusion models have the following problems in the specific domain of urban security anomaly generation: First, poor domain adaptability; pre-trained models lack domain knowledge of urban security anomalies, resulting in generated images that are not realistic in terms of anomaly form and scene context; second, inaccurate anomaly features; the generated anomalies deviate from reality in terms of location, scale, and severity; third, low training efficiency; training a diffusion model from scratch requires significant computational resources and time, and is prone to overfitting in small sample scenarios; and finally, an incomplete evaluation system: there is a lack of systematic evaluation methods for urban security anomaly generation tasks.

[0004] With the deepening of smart city construction, urban security systems based on video surveillance are becoming increasingly widespread. However, in practical applications, the scarcity, diverse forms, and complex scenarios of abnormal events (such as traffic accidents, fires, and crowd gatherings) severely restrict the performance of intelligent analysis models. Traditional data augmentation methods cannot generate new abnormal event samples with realistic semantics, while general generative models perform poorly in the field of urban security.

[0005] While there are some image generation methods based on GANs or diffusion models in the existing technology, they still suffer from problems such as low generation quality, insufficient detail representation, and poor training stability when dealing with the specific task of urban security anomalies.

[0006] Therefore, improving the quality and diversity of images generated for urban safety anomalies is a pressing technical problem that needs to be solved. Summary of the Invention

[0007] The technical objective of this invention is to provide a method, system, device, and medium for generating images of urban safety anomalies, in order to address the problem of how to improve the quality and diversity of generated images of urban safety anomalies.

[0008] The technical objective of this invention is achieved as follows: a method for generating images of urban safety anomalies, the method being as follows:

[0009] Construct and preprocess a dataset of urban safety anomalies: collect surveillance images containing normal scenes and various anomalies, construct a labeled map by annotating the location and category of anomalies in the surveillance images, and divide the training set and test set according to a preset ratio;

[0010] Perform heterogeneous feature fusion to generate comprehensive conditional features: encode the original monitoring image into latent features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal for guided generation;

[0011] Domain adaptation of pre-trained models based on intelligent fine-tuning framework: Load the pre-trained diffusion model U-Net, freeze the latent weights of the shared pre-trained diffusion model U-Net, and perform efficient fine-tuning using fused conditional features and multi-objective loss function;

[0012] Enhanced feature extraction by integrating multi-scale and spatial attention mechanisms: Multi-scale attention modules and spatial attention modules based on dilated convolution are introduced into the pre-trained diffusion model U-Net. Multi-scale semantic features and key spatial regions are extracted and focused respectively through the multi-scale attention module and spatial attention module based on dilated convolution.

[0013] Conditional image generation is performed: A comprehensive conditional feature x is generated using an input normal scene image, text prompts describing the target anomaly, and corresponding anomaly region annotations. t ; in conditional feature x t Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated.

[0014] A systematic evaluation framework is adopted to quantify the generation performance: the FID index is used to evaluate the realism of the generated image distribution, and the CLIP-I and CLIP-T indices are used to measure visual fidelity and text-image semantic alignment, respectively.

[0015] As a preferred approach, heterogeneous feature fusion involves aligning and fusing information sources representing different semantics, including scene content, anomaly location and type, and randomly generated content, in the latent space. This constructs an information-complete generation condition, guiding the generation of images that both conform to the scene context and accurately reflect anomaly features; specifically as follows:

[0016] Define the input surveillance image as X∈R H×W×3 The anomaly region annotation map corresponding to the monitoring image is M∈R H×W×1 The labeled image includes information on the location and category of defects. During processing, it can clearly identify the main anomaly type and its corresponding location, reducing interference from other irrelevant information. Here, H and W represent the height and width of the image, respectively.

[0017] The labeled image M undergoes a downsampling operation through a convolutional layer with a stride greater than 1. This reduces the spatial resolution of the labeled image to match the latent feature map, while compressing the data volume and extracting key spatial structure information to obtain the downsampled labeled image features f. mask ;

[0018] Simultaneously, the original surveillance image X is input into a pre-trained VQ-VAE encoder, which maps the original surveillance image X from a high-dimensional pixel space to a low-dimensional latent space, thereby obtaining latent features f that preserve the core semantics of the image. image ;

[0019] The discrete feature vectors encoded by the VQ-VAE encoder are encoded into a visual feature set. By finding the nearest word of the visual feature set in the visual codebook, the visual representation of the corresponding image is obtained based on the visual codebook. Simultaneously, the visual representation of the image maps any image to the representation space corresponding to the visual codebook. The Gaussian noise required during the diffusion process is defined as f. noisy ;

[0020] The annotation map feature f mask , latent features f image and f noisy Three feature tensors from different sources and with complementary semantics are concatenated along the channel dimension to form a comprehensive feature basis f0, in the form: f0 = concat(f image ,f mask ,f noisy ); where concat(·) represents a concatenation operation along the channel dimension;

[0021] The comprehensive feature basis f0 is processed through a 2D convolutional layer with a 1×1 kernel for feature fusion and nonlinear transformation. Nonlinearity is introduced through the ReLU activation function, and the final output is the input conditional feature x for the noise addition process of the diffusion model. t The form is: x t =ReLU(Conv2D(f0)); where ReLU(·) represents the ReLU neural network activation function; Conv2D(·) represents a 2D convolution operation with a kernel size of 1×1.

[0022] More preferably, the intelligent fine-tuning framework reduces computational costs by sharing weights when fine-tuning using a set number of real outlier samples, and introduces samples generated by the pre-trained latent diffusion model itself as supervision signals to maintain the generative diversity and prior knowledge of the latent diffusion model and prevent overfitting; specifically as follows:

[0023] Load a latent diffusion model pre-trained on general image data and freeze and share the latent layer weights of the encoder part of the latent diffusion model, which greatly reduces the number of trainable parameters in the fine-tuning process and reduces computational overhead.

[0024] Fine-tuning process on a small urban security anomaly dataset D train The process proceeds as follows: For each training iteration, in addition to calculating the reconstruction loss between the real sample x and the generated result, a self-generating supervision process is simultaneously initiated based on the intelligent fine-tuning framework. Specifically, the latent diffusion model receives a randomly initialized noise ∈ ~N(0,I) and a general text condition c. pr Generate a batch of samples x pr Then, construct a multi-objective loss function sharing optimization potential diffusion model.

[0025] More preferably, the target loss function consists of two parts, as follows:

[0026] One part is the subject reconstruction loss based on the text condition c of the real sample x and the specific identifier corresponding to the real sample x. The goal of the subject reconstruction loss is to enable the latent diffusion model to learn to accurately reconstruct the corresponding real sample x based on the text c containing the specific identifier. The real sample is the core of the model learning specific subject features.

[0027] The other part is Prior Preservation Loss, which aims to prevent the model from "learning new things and forgetting old ones" by making the latent diffusion model denoising using only textual conditions c. pr The original category image is generated, and its loss is calculated to constrain the potential diffusion model from deviating excessively from its original knowledge about the category, effectively mitigating overfitting and "language drift".

[0028] The total loss function is defined as follows:

[0029]

[0030] in, This represents a latent diffusion model (typically U-Net), where the inputs are a noisy latent representation and a condition c, and the goal is to predict the denoised image; t is any specific time step in the diffusion process; w tα represents the loss weighting coefficient at different time steps t, used to balance the importance of noise prediction error in different denoising stages; t and σ t λ is the noise scheduling parameter, which controls the noise intensity and signal retention ratio at step t; λ is the weighting coefficient that balances the contributions of the two losses. It is the L2 norm squared (MSE), used to measure the difference between the predicted value and the actual target value of the potential diffusion model; α′ t σ′ t w′ t Both ∈′ represent the corresponding parameters under the prior conditions; by minimizing the total loss function, the latent diffusion model is forced to maintain its original and diverse generative capabilities while learning anomalous features in a specific domain, thereby achieving fast, stable and effective domain transfer under small sample settings.

[0031] As a preferred embodiment, the multi-scale attention module based on dilated convolution is as follows:

[0032] A 1×1 convolution is used to perform channel aggregation on the input noisy feature map to achieve preliminary feature integration;

[0033] The feature maps after initial aggregation are input into three parallel dilated convolutional layers. The dilation rates of the three parallel dilated convolutional layers are set to 1, 2 and 3 respectively. Different dilation rates give each convolutional branch a different receptive field, so that local details, intermediate range and broader contextual features can be captured respectively.

[0034] Let the output feature corresponding to a dilated convolution with a dilation rate of k be... The features at the three scales are then spliced ​​and fused into X. fuse The format is:

[0035] A set of weighting coefficients w, automatically learned during the model training phase, is used. k The feature maps at different scales are weighted and accumulated to generate a first feature map containing multi-scale semantic information.

[0036] Preferably, the spatial attention module performs global average pooling on the input noisy feature map along the channel dimension, compressing a C-channel feature map into a compressed feature map with 1 channel that can represent global spatial information; then, a 5×5×1 convolution kernel is used to perform a convolution operation on the compressed feature map to calculate the importance of each spatial location, resulting in a spatial attention weight map; finally, the spatial attention weight map is multiplied element-wise with the original noisy feature map to enhance the features of key regions and output a second feature map with spatial attention.

[0037] As a preferred approach, a systematic evaluation framework is used to quantify the performance as follows:

[0038] The Fréchet Inception Distance (FID) is used as the core metric to evaluate the overall realism and distribution matching of the generated images. FID is achieved through a pre-trained Inception-v3 network, which processes a set of real images X. real and generate image set X gen Map each feature to a feature space, and assume that its feature distribution follows a multivariate Gaussian distribution. Calculate the real image set X. real and generate image set X gen The Fréchet distance (also known as the Wasserstein-2 distance) between Gaussian distributions is calculated using the following formula:

[0039]

[0040] Where, μ real and μ gen These are the mean vectors of the true features and the generated features, respectively; ∑ real and ∑ gen It is the covariance matrix of real features and generated features; Tr represents the trace of the matrix; the lower the FID value, the closer the distribution of the generated image is to the distribution of the real image, that is, the higher the overall generation quality.

[0041] The CLIP-I (CLIP Image Similarity) metric is introduced to measure the specific similarity in visual features between generated and real images. The CLIP-I metric utilizes the image encoder E of a CLIP-I model pre-trained on large-scale image-text pairs. I For each real image I real and its corresponding generated image I gen Extract the corresponding image features and embed them into v real =E I (I real ) and v gen =E I (I gen ), calculate v real With v gen The cosine similarity between the images is calculated and averaged over all image pairs, using the following formula:

[0042]

[0043] The higher the CLIP-I value, the closer the generated image is to the real reference image in terms of visual features, and the better the fidelity.

[0044] The CLIP-T (CLIP Text-Image Similarity) metric is introduced to evaluate whether the generated image accurately matches the given text description, i.e., the accuracy of conditional control. The CLIP-T metric utilizes the image encoder E of the CLIP model. I and text encoder E T For each generated image I gen And its corresponding text prompt T, respectively, extract image features and embed v gen =E I (I gen ) and text feature embedding t=E T (T), then calculate the cosine similarity between the image feature embedding and the text feature embedding, and take the average over all samples, as shown in the following formula:

[0045]

[0046] The higher the CLIP-T value, the better the semantic alignment between the generated image and the input text, and the stronger the conditional control capability.

[0047] The FID, CLIP-I, and CLIP-T metrics are used to construct a systematic evaluation matrix. The performance of the generative model is comprehensively and objectively evaluated from three key dimensions: statistical distribution, visual detail, and semantic control, ensuring the comprehensiveness and scientific nature of the evaluation results.

[0048] A system for generating images of urban safety anomalies, comprising the following components:

[0049] The dataset construction and preprocessing unit is used to collect surveillance images containing normal scenes and various abnormal events, construct annotation maps by labeling the location and category of abnormal areas in the surveillance images, and divide the training set and test set according to a preset ratio.

[0050] The heterogeneous feature fusion unit is used to encode the original monitoring image into latent features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal for guided generation.

[0051] Based on the intelligent fine-tuning framework unit, it is used to load the pre-trained diffusion model U-Net, freeze the latent weights of the shared pre-trained diffusion model U-Net, and use the fused conditional features and multi-objective loss function for efficient fine-tuning;

[0052] The feature extraction unit is used to introduce a multi-scale attention module and a spatial attention module based on dilated convolution in the pre-trained diffusion model U-Net. The multi-scale attention module and the spatial attention module based on dilated convolution extract multi-scale semantic features and focus on key spatial regions, respectively.

[0053] The image generation module is used to generate comprehensive conditional features x by using an input normal scene image, text prompts describing the target anomaly, and corresponding anomaly region annotation maps. t ; in conditional feature x t Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated.

[0054] The performance evaluation module is used to calculate the distribution distance between the generated image set and the real image set in the Inception-v3 feature space using the FID metric, thereby evaluating the overall quality of the generated images; it is also used to calculate the cosine similarity between the generated images and the corresponding real images in the CLIP feature space using the CLIP-I metric, thereby evaluating the consistency of visual features; and it is further used to calculate the semantic alignment between the generated images and the input text prompts in the CLIP cross-modal space using the CLIP-T metric, thereby evaluating the control accuracy of conditional generation.

[0055] An electronic device includes: a memory and at least one processor;

[0056] The memory contains computer programs;

[0057] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the urban security anomaly image generation method as described above.

[0058] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the urban security anomaly image generation method described above.

[0059] The urban safety anomaly image generation method, system, device, and medium of the present invention have the following advantages:

[0060] (I) This invention constructs a dedicated urban safety dataset containing multiple anomaly types; then, it proposes a heterogeneous feature fusion method, which splices and fuses the downsampled anomaly region annotation map, the latent features of the original monitoring image, and the latent features of Gaussian noise in the channel dimension, enhancing the model's ability to perceive details of anomaly events; further, it designs a fine-tuning framework based on intelligent fusion of anomaly condition information, achieving efficient adaptation to the pre-trained diffusion model through shared latent weights and self-generated sample supervision; finally, it establishes a systematic experimental evaluation system, verifying the superiority of the method on indicators such as FID, CLIP-I, and CLIP-T. This invention significantly improves the generation quality and diversity of urban safety anomaly event images through innovative feature fusion strategies and fine-tuning mechanisms, providing effective technical support for expanding training data for intelligent security systems, and is particularly suitable for scenarios such as traffic management, public safety monitoring, and emergency response;

[0061] (II) Through heterogeneous feature fusion, the model of this invention can generate urban safety incident images with rich details and accurate anomaly features, reducing the FID index by more than 50% compared with existing methods, thus significantly improving the generation quality.

[0062] (III) The feature fusion and fine-tuning strategy of this invention is specifically designed for the characteristics of abnormal urban security events, so that the generated images are more in line with the needs of real monitoring scenarios and enhance domain adaptability;

[0063] (iv) This invention establishes a multi-index evaluation system to comprehensively and objectively measure the generation performance, providing a clear direction for method optimization and improving the evaluation system;

[0064] (v) The high-quality abnormal event samples generated by this invention can effectively expand the training dataset, improve the performance of downstream anomaly detection models, have important public safety value, and have outstanding practical value.

[0065] (vi) This invention is deployed on an urban security monitoring platform and connects with the existing video surveillance system to expand the training dataset by using the generated abnormal event samples, thereby improving the recognition accuracy and robustness of the downstream anomaly detection model in real scenarios.

[0066] (vii) This invention solves the problems of poor domain adaptability, inaccurate abnormal features, low training efficiency, and imperfect evaluation system in existing urban safety anomaly image generation methods by processing the entire process of dataset construction, heterogeneous feature fusion, model fine-tuning, attention feature enhancement, conditional image generation, and systematic performance evaluation.

[0067] (viii) This invention ensures that the generative model can grasp the global scene layout of abnormal events and finely depict its local structural details by working together with a multi-scale attention module and a spatial attention module based on dilated convolution. Attached Figure Description

[0068] The invention will be further described below with reference to the accompanying drawings.

[0069] Appendix Figure 1 A schematic diagram of the overall framework for a method to generate images of urban safety anomalies;

[0070] Appendix Figure 2 This is a schematic diagram of feature enhancement based on a multi-scale attention mechanism;

[0071] Appendix Figure 3 This is a schematic diagram of model fine-tuning;

[0072] Appendix Figure 4 A flowchart of a method for generating images of urban safety anomalies. Detailed Implementation

[0073] The method, system, device and medium for generating images of urban safety anomalies of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] Example 1:

[0075] As attached Figure 4 As shown in the figure, this embodiment provides a method for generating images of urban safety anomalies, which is as follows:

[0076] S1. Construct and preprocess the urban safety anomaly event dataset: collect surveillance images containing normal scenes and various anomalies, construct a labeled map by labeling the location and category of the anomalies in the surveillance images, and divide the training set and test set according to a preset ratio;

[0077] S2. Perform heterogeneous feature fusion to generate comprehensive conditional features: Encode the original monitoring image into latent features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal generated under guidance.

[0078] S3. Domain adaptation of pre-trained models based on intelligent fine-tuning framework: Load the pre-trained diffusion model U-Net, freeze the latent weights of the shared pre-trained diffusion model U-Net, and use the fused conditional features and multi-objective loss function for efficient fine-tuning.

[0079] S4. Enhanced Feature Extraction by Integrating Multi-Scale and Spatial Attention Mechanisms: In the pre-trained diffusion model U-Net, a multi-scale attention module and a spatial attention module based on dilated convolution are introduced. Multi-scale semantic features and key spatial regions are extracted and focused respectively through the multi-scale attention module and the spatial attention module based on dilated convolution.

[0080] S5. Execution of condition-guided image generation: This involves generating a comprehensive conditional feature x using a normal scene image as input, text prompts describing the target anomaly, and corresponding anomaly region annotations. t ; in conditional feature x t Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated.

[0081] S6. Employ a systematic evaluation system to quantify generation performance: Use the FID index to evaluate the authenticity of the generated image distribution, and use CLIP-I and CLIP-T indices to measure visual fidelity and text-image semantic alignment, respectively.

[0082] As attached Figure 1 As shown, the heterogeneous feature fusion in step S2 of this embodiment involves aligning and fusing information sources representing different semantics, including scene content, anomaly location and type, and randomly generated content, in the latent space to construct a complete generation condition, guiding the generation of an image that both conforms to the scene context and accurately reflects the anomaly features; specifically as follows:

[0083] S201. Define the input monitoring image as X∈R H×W×3 The anomaly region annotation map corresponding to the monitoring image is M∈R H ×W×1 The labeled image includes information on the location and category of defects. During processing, it can clearly identify the main anomaly type and its corresponding location, reducing interference from other irrelevant information. Here, H and W represent the height and width of the image, respectively.

[0084] S202. The labeled image M undergoes a convolutional layer downsampling operation with a stride greater than 1. This reduces the spatial resolution of the labeled image to match the latent feature map, while compressing the data volume and extracting key spatial structure information to obtain the downsampled labeled image features f. mask ;

[0085] S203. Simultaneously, the original surveillance image X is input into a pre-trained VQ-VAE encoder to map the original surveillance image X from a high-dimensional pixel space to a low-dimensional latent space, thereby obtaining latent features f that preserve the core semantics of the image. image ;

[0086] S204. Encode the discrete feature vectors encoded by the VQ-VAE encoder into a visual feature set. By finding the nearest word of the visual feature set in the visual codebook, obtain the visual representation of the corresponding image according to the visual codebook. At the same time, the visual representation of the image maps any image to the representation space corresponding to the visual codebook, and defines the Gaussian noise required in the diffusion process as f. noisy ;

[0087] S205, Add the feature f of the labeled map mask , latent features f image and f noisy Three feature tensors from different sources and with complementary semantics are concatenated along the channel dimension to form a comprehensive feature basis f0, in the form: f0 = concat(f image ,f mask ,f noisy ); where concat(·) represents a concatenation operation along the channel dimension;

[0088] S206. The comprehensive feature basis f0 is processed through a 2D convolutional layer with a 1×1 kernel for feature fusion and nonlinear transformation. Nonlinearity is introduced through the ReLU activation function, and the final output is the input conditional feature x of the diffusion model's noise addition process. t The form is: x t =ReLU(Conv2D(f0)); where ReLU(·) represents the ReLU neural network activation function; Conv2D(·) represents a 2D convolution operation with a kernel size of 1×1. In this way, the model can simultaneously perceive "what the scene background is", "where the anomaly occurs", and "what random changes are needed for generation" in each step of denoising generation, thereby significantly improving the accuracy of abnormal events in the generated image and the realism of the overall scene.

[0089] As attached Figure 2 As shown, in step S3 of this embodiment, the intelligent fine-tuning framework reduces computational costs by sharing weights when fine-tuning using a set number of real abnormal samples, and introduces samples generated by the pre-trained latent diffusion model itself as supervision signals to maintain the generation diversity and prior knowledge of the latent diffusion model and prevent overfitting; specifically as follows:

[0090] S301. Load a latent diffusion model pre-trained on general image data, and freeze and share the latent layer weights of the encoder part of the latent diffusion model, which greatly reduces the number of trainable parameters in the fine-tuning process and reduces computational overhead.

[0091] S302, Fine-tuning process in a small urban security anomaly dataset D train The process proceeds as follows: For each training iteration, in addition to calculating the reconstruction loss between the real sample x and the generated result, a self-generating supervision process is simultaneously initiated based on the intelligent fine-tuning framework. Specifically, the latent diffusion model receives a randomly initialized noise ∈ ~N(0,I) and a general text condition c. pr Generate a batch of samples x pr Then, construct a multi-objective loss function sharing optimization potential diffusion model.

[0092] The target loss function in step S302 of this embodiment includes two parts, as follows:

[0093] One part is the subject reconstruction loss based on the text condition c of the real sample x and the specific identifier corresponding to the real sample x. The goal of the subject reconstruction loss is to enable the latent diffusion model to learn to accurately reconstruct the corresponding real sample x based on the text c containing the specific identifier. The real sample is the core of the model learning specific subject features.

[0094] The other part is Prior Preservation Loss, which aims to prevent the model from "learning new things and forgetting old ones" by making the latent diffusion model denoising using only textual conditions c. pr The original category image is generated, and its loss is calculated to constrain the potential diffusion model from deviating excessively from its original knowledge about the category, effectively mitigating overfitting and "language drift".

[0095] The total loss function is defined as follows:

[0096]

[0097] in, This represents a latent diffusion model (typically U-Net), where the inputs are a noisy latent representation and a condition c, and the goal is to predict the denoised image; t is any specific time step in the diffusion process; w t α represents the loss weighting coefficient at different time steps t, used to balance the importance of noise prediction error in different denoising stages; t and σ t λ is the noise scheduling parameter, which controls the noise intensity and signal retention ratio at step t; λ is the weighting coefficient that balances the contributions of the two losses. It is the L2 norm squared (MSE), used to measure the difference between the predicted value and the actual target value of the potential diffusion model; α ′ t σ t ′ w t ′ and ∈ ′ All represent the corresponding parameters under prior conditions; by minimizing the total loss function, the latent diffusion model is forced to maintain its original and diverse generative capabilities while learning anomalous features in a specific domain, thereby achieving fast, stable and effective domain transfer under small sample settings.

[0098] The feature enhancement based on the multi-scale attention mechanism in step S4 of this embodiment is a key technology for improving the model's ability to perceive and extract features of abnormal events. (See attached...) Figure 3 As shown, its core idea is to enable the model to adaptively capture anomalous targets of different sizes and focus on key spatial regions through parallel multi-scale convolution and spatial weight recalibration, thereby enhancing the detail richness and structural accuracy of the generated image.

[0099] The multi-scale attention module based on dilated convolution in step S4 of this embodiment is as follows:

[0100] S4-101. A 1×1 convolution is used to perform channel aggregation on the input noise feature map to achieve preliminary feature integration.

[0101] S4-102. Input the initially aggregated feature maps into three parallel dilated convolutional layers respectively. Set the dilation rates of the three parallel dilated convolutional layers to 1, 2 and 3 respectively. Different dilation rates give each convolutional branch a different receptive field, so as to capture local details, medium range and broader contextual features respectively.

[0102] S4-103. Let the output feature corresponding to a dilated convolution with a dilation rate of k be... The features at the three scales are then spliced ​​and fused into X. fuse The format is:

[0103] S4-104. Use a set of weighting coefficients w that are automatically learned during the model training phase. k The feature maps at different scales are weighted and accumulated to generate a first feature map containing multi-scale semantic information.

[0104] In step S4 of this embodiment, the spatial attention module performs global average pooling on the input noise feature map along the channel dimension, compressing a C-channel feature map into a compressed feature map with 1 channel that can represent global spatial information. Then, a 5×5×1 convolution kernel is used to perform a convolution operation on the compressed feature map to calculate the importance of each spatial location, resulting in a spatial attention weight map. Finally, the spatial attention weight map is multiplied element-wise with the original noise feature map to enhance the features of key regions and output a second feature map with spatial attention.

[0105] All experiments in this embodiment were conducted on a single computer equipped with an NVIDIA GeForce RTX4090 GPU and The system utilizes a W-2150B CPU and has 24GB of video memory. The environment configuration is shown in Table 1. The experimental environment was built using PyTorch 1.12.0 and Python version 3.10.10, providing a stable computing platform for subsequent model training and debugging. PyTorch is an open-source framework widely used in deep learning, offering flexible tensor operations, automatic differentiation mechanisms, and rich model building and training tools.

[0106] Table 1 Experimental Environment Configuration

[0107]

[0108]

[0109] This embodiment conducts experimental analysis. The dataset used in this embodiment consists of 256×256 scene defect samples obtained through self-creation and cropping. The training set contains 360 images, and the test set contains 48 images. To verify the visual realism and semantic consistency of the model's generation capabilities, this embodiment mainly adopts three indicators: FID, CLIP-I, and CLIP-T. A systematic experimental evaluation system is a key technology for objectively quantifying and verifying the quality and semantic consistency of generated images. Its core idea is to abandon single subjective evaluations and adopt a set of multi-dimensional, computable objective indicators to comprehensively evaluate from three aspects: image realism, visual fidelity, and text-image alignment, providing an accurate and reproducible performance benchmark for model optimization.

[0110] First, FID (Fréchet Inception Distance) is used as the core metric to evaluate the overall realism and distribution matching of the generated images. FID is applied to a pre-trained Inception-v3 network using a set of real images X. real and generate image set X gen Map each feature to a feature space, and assume that its feature distribution follows a multivariate Gaussian distribution. Calculate the real image set X. real and generate image set X gen The Fréchet distance (also known as the Wasserstein-2 distance) between Gaussian distributions is calculated using the following formula:

[0111] FID = ||μ real -μ gen || 2 +Tr(∑ real ∑ gen -2(∑ real ∑ gen ) 1 / 2 );

[0112] Where, μ realand μ gen These are the mean vectors of the true features and the generated features, respectively; ∑ real and ∑ gen It is the covariance matrix of real features and generated features; Tr represents the trace of the matrix; the lower the FID value, the closer the distribution of the generated image is to the distribution of the real image, that is, the higher the overall generation quality.

[0113] Secondly, the CLIP-I (CLIP Image Similarity) metric is introduced to measure the specific similarity in visual features between the generated image and the real image. The CLIP-I metric utilizes the image encoder E of the CLIP-I model pre-trained on large-scale image-text pairs. I For each real image I real and its corresponding generated image I gen Extract the corresponding image features and embed them into v real =E I (I real ) and v gen =E I (I gen ), calculate v real With v gen The cosine similarity between the images is calculated and averaged over all image pairs, using the following formula:

[0114]

[0115] The higher the CLIP-I value, the closer the generated image is to the real reference image in terms of visual features, and the better the fidelity.

[0116] Finally, the CLIP-T (CLIP Text-Image Similarity) metric is introduced to evaluate whether the generated image accurately matches the given text description, i.e., the accuracy of conditional control. The CLIP-T metric utilizes the image encoder E of the CLIP model. I and text encoder E T For each generated image I gen And its corresponding text prompt T, respectively, extract image features and embed v gen =E I (I gen ) and text feature embedding t=E T (T), then calculate the cosine similarity between the image feature embedding and the text feature embedding, and take the average over all samples, as shown in the following formula:

[0117]

[0118] The higher the CLIP-T value, the better the semantic alignment between the generated image and the input text, and the stronger the conditional control capability.

[0119] The FID, CLIP-I, and CLIP-T metrics are used to construct a systematic evaluation matrix. The performance of the generative model is comprehensively and objectively evaluated from three key dimensions: statistical distribution, visual detail, and semantic control, ensuring the comprehensiveness and scientific nature of the evaluation results.

[0120] To verify the effectiveness of the added modules, ablation experiments were conducted under the same experimental environment to compare the two innovative approaches: heterogeneous feature fusion and feature enhancement based on multi-scale attention mechanisms. This was done to verify the effectiveness and feasibility of the methods. Taking the scenario of insulation cover detachment as an example of anomalies, the impact of each module on the algorithm was analyzed from the perspectives of three evaluation metrics: FID, CLIP-I, and CLIP-T. The experimental results are shown in Table 2. In general, the algorithm without adding two modules showed the worst performance. Adding the heterogeneous feature fusion method alone reduced FID performance by 1.4016, while CLIP-I and CLIP-T performance increased by 0.0684 and 0.0508, respectively. Adding feature enhancement based on multi-scale attention mechanism alone reduced FID performance by 0.5086, while CLIP-I and CLIP-T performance increased by 0.0729 and 0.0531, respectively. The algorithm with both modules added showed the best performance. Compared with the model without any modules added, FID performance decreased by 3.8175, while CLIP-I and CLIP-T performance increased by 0.1667 and 0.1299, respectively.

[0121] Table 2 Ablation Experiment Results

[0122]

[0123]

[0124] Example 2:

[0125] This embodiment provides a system for generating images of urban safety anomalies. This system implements the method for generating images of urban safety anomalies as described in Embodiment 1. The system includes:

[0126] The dataset construction and preprocessing unit is used to collect surveillance images containing normal scenes and various abnormal events, construct annotation maps by labeling the location and category of abnormal areas in the surveillance images, and divide the training set and test set according to a preset ratio.

[0127] The heterogeneous feature fusion unit is used to encode the original monitoring image into latent features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal for guided generation.

[0128] Based on the intelligent fine-tuning framework unit, it is used to load the pre-trained diffusion model U-Net, freeze the latent weights of the shared pre-trained diffusion model U-Net, and use the fused conditional features and multi-objective loss function for efficient fine-tuning;

[0129] The feature extraction unit is used to introduce a multi-scale attention module and a spatial attention module based on dilated convolution in the pre-trained diffusion model U-Net. The multi-scale attention module and the spatial attention module based on dilated convolution extract multi-scale semantic features and focus on key spatial regions, respectively.

[0130] The image generation module is used to generate comprehensive conditional features x by using an input normal scene image, text prompts describing the target anomaly, and corresponding anomaly region annotation maps. t ; in conditional feature x t Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated.

[0131] The performance evaluation module is used to calculate the distribution distance between the generated image set and the real image set in the Inception-v3 feature space using the FID metric, thereby evaluating the overall quality of the generated images; it is also used to calculate the cosine similarity between the generated images and the corresponding real images in the CLIP feature space using the CLIP-I metric, thereby evaluating the consistency of visual features; and it is further used to calculate the semantic alignment between the generated images and the input text prompts in the CLIP cross-modal space using the CLIP-T metric, thereby evaluating the control accuracy of conditional generation.

[0132] In this embodiment, the dataset construction and preprocessing unit constructs a dataset from monitoring images containing normal scenes and abnormal events, defining a single monitoring image as X∈R. H×W×3 , where H and W represent the height and width of the image, respectively.

[0133] In this embodiment, the heterogeneous feature fusion unit uses a pre-trained VQ-VAE encoder to encode the original surveillance image into latent features f. image The anomaly annotation map is downsampled using a convolutional layer with a stride greater than 1 to obtain a feature f with the same size as the latent feature space. mask ; will f image f mask With Gaussian noise characteristics f noisy Concatenate along the channel dimension to obtain the fused feature f0 = concat(f image ,f mask ,f noisy Finally, a 1×1 convolutional layer and a ReLU activation function are used to perform a nonlinear transformation on the fused features, outputting the final conditional feature x. t=ReLU(Conv2D(f0)).

[0134] The intelligent fine-tuning framework in this embodiment specifically includes: loading a pre-trained latent diffusion model onto a general image and text dataset; employing a weight-sharing strategy to freeze the latent weights of the model's encoder part; and combining the fused conditional features x t The textual conditional features are injected into the U-Net structure of the model; a multi-objective loss function L, which includes self-generated sample supervision, is used. total Optimize the training, where the loss function is defined as follows:

[0135] In this embodiment, the feature extraction unit uses a multi-scale attention module to perform channel aggregation on the input features using a 1×1 convolution. Then, it extracts multi-scale features through three parallel dilated convolutional layers with dilation rates of 1, 2, and 3, respectively. Finally, it fuses the features using learnable weighting coefficients to obtain the first feature map. The spatial attention module performs global average pooling on the feature map along the channel dimension to obtain channel compressed features. It uses a 5×5×1 convolution to calculate spatial importance weights and multiplies the obtained spatial attention weights with the original feature map to generate a second feature map with spatial attention.

[0136] The image generation module in this embodiment includes an input normal scene image, a text prompt describing the target anomaly, and a corresponding anomaly region annotation map; a comprehensive conditional feature x is generated through a heterogeneous feature fusion module. t ; in conditional feature x t Guided by text embedding, a multi-step reverse denoising process of the diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated.

[0137] Example 3:

[0138] This embodiment also provides an electronic device, including: a memory and a processor;

[0139] The memory stores the instructions executed by the computer.

[0140] The processor executes computer execution instructions stored in the memory, causing the processor to execute the urban safety anomaly image generation method in any embodiment of the present invention.

[0141] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0142] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0143] Example 4:

[0144] This embodiment also provides a computer-readable storage medium storing multiple instructions, which are loaded by a processor to cause the processor to execute the urban safety anomaly image generation method of any embodiment of the present invention. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0145] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0146] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0147] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0148] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating images of urban safety anomalies, characterized in that, The method is as follows: Construct and preprocess a dataset of urban safety anomalies: collect surveillance images containing normal scenes and various anomalies, construct a labeled map by annotating the location and category of anomalies in the surveillance images, and divide the training set and test set according to a preset ratio; Perform heterogeneous feature fusion to generate comprehensive conditional features: encode the original monitoring image into shallow features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal for guided generation. Domain adaptation of pre-trained models based on intelligent fine-tuning framework: Load the pre-trained diffusion model U-Net, freeze the shallow weights of the shared pre-trained diffusion model U-Net, and perform efficient fine-tuning using fused conditional features and multi-objective loss function; Enhanced feature extraction by integrating multi-scale and spatial attention mechanisms: Multi-scale attention modules and spatial attention modules based on dilated convolution are introduced into the pre-trained diffusion model U-Net. Multi-scale semantic features and key spatial regions are extracted and focused respectively through the multi-scale attention module and spatial attention module based on dilated convolution. Condition-guided image generation: This method generates comprehensive conditional features by using a normal scene image as input, text prompts describing the target anomaly, and corresponding anomaly region annotations. ; in conditional features Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated. A systematic evaluation framework is adopted to quantify the generation performance: the FID index is used to evaluate the realism of the generated image distribution, and the CLIP-I and CLIP-T indices are used to measure visual fidelity and text-image semantic alignment, respectively.

2. The method for generating images of urban safety anomalies according to claim 1, characterized in that, Heterogeneous feature fusion aligns and fuses information sources representing different semantics, including scene content, anomaly location and type, and randomly generated content, in a shallow spatial layer. This constructs an information-complete generation condition, guiding the generation of images that both conform to the scene context and accurately reflect anomaly features; specifically as follows: Define the input monitoring image as The abnormal area annotation map corresponding to the monitoring image is as follows: The labeled image includes information on the location and category of defects; where H and W represent the height and width of the image, respectively. Annotated diagram After performing a downsampling operation through a convolutional layer with a stride greater than 1, the spatial resolution of the labeled map is reduced to match that of the shallow feature map. At the same time, the data volume is compressed and key spatial structure information is extracted to obtain the features of the downsampled labeled map. ; At the same time, the original surveillance images The raw surveillance image is input into a pre-trained VQ-VAE encoder. Mapping from a high-dimensional pixel space to a low-dimensional shallow space to obtain shallow features that preserve the core semantics of the image. ; The discrete feature vectors encoded by the VQ-VAE encoder are encoded into a visual feature set. By finding the nearest word in the visual codebook to the visual feature set, the visual representation of the corresponding image is obtained based on the visual codebook. Simultaneously, the visual representation of the image maps any image to the representation space corresponding to the visual codebook. The Gaussian noise required during the diffusion process is defined as... ; Annotation map features Shallow features as well as Three feature tensors from different sources and with complementary semantics are concatenated along the channel dimension to form a comprehensive feature base. The format is: ;in, This indicates a concatenation operation along the channel dimension; Comprehensive characteristic base Feature fusion and nonlinear transformation are performed after passing through a 2D convolutional layer with a 1×1 kernel. Nonlinearity is introduced through the ReLU activation function, and the final output is the input conditional feature of the diffusion model's noise addition process. The format is: ;in, Represents the ReLU neural network activation function; This represents a 2D convolution operation with a kernel size of 1×1.

3. The method for generating images of urban safety anomalies according to claim 1 or 2, characterized in that, The intelligent fine-tuning framework reduces computational costs by sharing weights when fine-tuning using a set number of real outlier samples, and introduces samples generated by the pre-trained latent diffusion model itself as supervision signals to maintain the generative diversity and prior knowledge of the latent diffusion model, thus preventing overfitting; specifically as follows: Load a pre-trained latent diffusion model on general image data and freeze and share the shallow weights of the encoder part of the latent diffusion model; Fine-tuning process on a small urban security anomaly dataset This is performed on the above, and for each training iteration, in addition to calculating the real samples... In addition to the reconstruction loss between the generated results, a self-generating supervision process is simultaneously initiated based on the intelligent fine-tuning framework. Specifically, the latent diffusion model receives randomly initialized noise. A general text condition Generate a batch of samples Then, construct a multi-objective loss function sharing optimization potential diffusion model.

4. The method for generating images of urban safety anomalies according to claim 3, characterized in that, The target loss function consists of two parts, as follows: Part of it is based on real samples and real samples The subject reconstruction loss is the text condition c corresponding to a specific identifier. The goal of the subject reconstruction loss is to enable the latent diffusion model to learn to accurately reconstruct the corresponding real sample x based on the text c containing the specific identifier. The real sample is the core of the model's learning of specific subject features. The other part is the class prior retention loss, which aims to prevent the model from "learning new things and forgetting old ones" by enabling the latent diffusion model to simultaneously denoise using only textual conditions. The original category image is generated, and its loss is calculated to constrain the potential diffusion model from deviating excessively from its original knowledge about the category, effectively mitigating overfitting and "language drift". The total loss function is defined as follows: ； in, Let represent the latent diffusion model, whose inputs are the noisy latent representation and the condition c, and whose goal is to predict the denoised image; t is any specific time step in the diffusion process. This represents the loss weighting coefficient at different time steps t, used to balance the importance of noise prediction error in different denoising stages; and These are noise scheduling parameters that control the noise intensity and signal retention ratio at step t. It is a weighting coefficient that balances the contributions of the two losses; It is the L2 norm squared, used to measure the difference between the predicted value and the actual target value of the potential diffusion model; , , and All represent the corresponding parameters under prior conditions; by minimizing the total loss function, the latent diffusion model is forced to maintain its original and diverse generative capabilities while learning anomalous features in a specific domain, thereby achieving fast, stable and effective domain transfer under small sample settings.

5. The method for generating images of urban safety anomalies according to claim 1, characterized in that, The multi-scale attention module based on dilated convolution is as follows: A 1×1 convolution is used to perform channel aggregation on the input noisy feature map to achieve preliminary feature integration; The feature maps after initial aggregation are input into three parallel dilated convolutional layers. The dilation rates of the three parallel dilated convolutional layers are set to 1, 2 and 3 respectively. Different dilation rates give each convolutional branch a different receptive field, so that local details, intermediate range and broader contextual features can be captured respectively. Let the output feature corresponding to a dilated convolution with a dilation rate of k be... The features at the three scales are then spliced ​​and fused together. The format is: ; A set of weighting coefficients learned automatically during the model training phase is used. The feature maps at different scales are weighted and accumulated to generate a first feature map containing multi-scale semantic information.

6. The method for generating images of urban safety anomalies according to claim 1, characterized in that, The spatial attention module performs global average pooling on the input noisy feature map along the channel dimension, compressing a C-channel feature map into a compressed feature map with 1 channel that can represent global spatial information. Then, a 5×5×1 convolution kernel is used to perform a convolution operation on the compressed feature map to calculate the importance of each spatial location, resulting in a spatial attention weight map. Finally, the spatial attention weight map is multiplied element-wise with the original noisy feature map to enhance the features of key regions and output a second feature map with spatial attention.

7. The method for generating images of urban safety anomalies according to claim 1, characterized in that, The performance is quantified using a systematic evaluation framework as follows: The Full Image ID (FID) is used as the core metric to evaluate the overall realism and distribution matching of the generated images. The FID is processed through a pre-trained Inception-v3 network, which uses a set of real images... and generating image sets Map each feature to a feature space, and assume that its feature distribution follows a multivariate Gaussian distribution, then calculate the real image set. and generating image sets The Fréchet distance between Gaussian distributions is calculated using the following formula: ； in, and These are the mean vectors of the real features and the generated features, respectively; and It is the covariance matrix of the real features and the generated features; The FID value represents the trace of the matrix; the lower the FID value, the closer the distribution of the generated image is to the distribution of the real image, that is, the higher the overall generation quality. The CLIP-I metric is introduced to measure the specific similarity in visual features between generated and real images. The CLIP-I metric utilizes an image encoder based on a CLIP-I model pre-trained on large-scale image-text pairs. For each real image and its corresponding generated image Extract corresponding image features and embed them respectively and ,calculate and The cosine similarity between the images is calculated and averaged over all image pairs, using the following formula: ； The higher the CLIP-I value, the closer the generated image is to the real reference image in terms of visual features, and the better the fidelity. The CLIP-T metric is introduced to evaluate whether the generated image accurately conforms to the given text description, i.e., the accuracy of conditional control. The CLIP-T metric utilizes the image encoder of the CLIP model. and text encoder For each generated image Image features are extracted and embedded for each text prompt T and its corresponding text prompt T. and text feature embedding Then, the cosine similarity between the image feature embedding and the text feature embedding is calculated and averaged over all samples, as shown in the following formula: ； The higher the CLIP-T value, the better the semantic alignment between the generated image and the input text, and the stronger the conditional control capability. The FID, CLIP-I, and CLIP-T metrics are used to construct a systematic evaluation matrix. The performance of the generative model is comprehensively and objectively evaluated from three key dimensions: statistical distribution, visual detail, and semantic control, ensuring the comprehensiveness and scientific nature of the evaluation results.

8. A system for generating images of urban safety anomalies, characterized in that, This system is used to implement the urban safety anomaly image generation method as described in any one of claims 1 to 7; the system includes: The dataset construction and preprocessing unit is used to collect surveillance images containing normal scenes and various abnormal events, construct annotation maps by labeling the location and category of abnormal areas in the surveillance images, and divide the training set and test set according to a preset ratio. The heterogeneous feature fusion unit is used to encode the original surveillance image into shallow features, downsample the labeled image, and splice and fuse it with Gaussian noise features in the channel dimension to form a strong conditional signal for guided generation. Based on the intelligent fine-tuning framework unit, it is used to load the pre-trained diffusion model U-Net, freeze the shallow weights of the shared pre-trained diffusion model U-Net, and use the fused conditional features and multi-objective loss function for efficient fine-tuning. The feature extraction unit is used to introduce a multi-scale attention module and a spatial attention module based on dilated convolution in the pre-trained diffusion model U-Net. The multi-scale attention module and the spatial attention module based on dilated convolution extract multi-scale semantic features and focus on key spatial regions, respectively. The image generation module is used to generate comprehensive conditional features by using an input normal scene image, text prompts describing the target anomaly, and corresponding anomaly region annotation maps. ; in conditional features Guided by text embedding, a multi-step reverse denoising process of the latent diffusion model is executed; by gradually removing noise and preserving semantic features, a high-quality abnormal event image that conforms to the text description and spatial location constraints is finally generated. The performance evaluation module is used to calculate the distribution distance between the generated image set and the real image set in the Inception-v3 feature space using the FID metric, thereby evaluating the overall quality of the generated images; it is also used to calculate the cosine similarity between the generated images and the corresponding real images in the CLIP feature space using the CLIP-I metric, thereby evaluating the consistency of visual features; and it is further used to calculate the semantic alignment between the generated images and the input text prompts in the CLIP cross-modal space using the CLIP-T metric, thereby evaluating the control accuracy of conditional generation.

9. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the urban security anomaly image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method for generating images of urban security anomalies as described in any one of claims 1 to 7.

Images