A method and system for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception.

Abstract

This invention discloses a method and system for generating UAV accident images that integrates multimodal semantic constraints and camera physical perception. The method includes: generating soft semantic masks and class prototype libraries through panoramic segmentation and open vocabulary alignment to achieve joint spatial and category constraints on traffic scenes; constructing an encoder-decoder network combining content and global style, and utilizing a semantically modulated cross-attention normalization module to achieve deep information fusion; extracting sensor style codes and camera responses from real UAV images to achieve precise style and content injection and modulation; employing a panoramic conditional multi-expert discriminator to ensure the geometric and semantic consistency of the generated images, and introducing a hybrid strategy of adversarial loss and dual ranking loss to improve the quality and stability of the generated images. This invention not only improves the performance of simulated images but also provides a new solution for the dynamic generation of complex traffic scenes.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation systems and computer vision, specifically to a method and system for generating UAV accident images that integrates multimodal semantic constraints and camera physical perception. Background Technology

[0002] Drones are playing an increasingly important role in intelligent transportation systems, offering unique observational advantages for traffic accident monitoring and emergency response from their high-altitude perspective. However, this perspective also introduces specific technical challenges: limited by battery life and airspace control, real drones struggle to continuously collect large-scale, high-quality traffic accident image data, especially for sudden, low-frequency traffic accident scenarios. The scarcity of data severely restricts the training effectiveness of deep learning-based detection and recognition models. Existing solutions primarily rely on simulation platforms to generate synthetic data. While such methods can generate images in batches, significant inter-domain differences exist between their output and real drone images, manifesting as distorted texture details, unnatural lighting, and inconsistent object geometry. This results in insufficient generalization ability of models trained on synthetic data in real-world scenarios.

[0003] In recent years, generative artificial intelligence technologies such as generative adversarial networks (GANs) and diffusion models have been applied to image style transfer and data augmentation tasks, attempting to transform simulated images into more realistic drone-view images. However, under the unique overhead perspective of drones, existing methods still face multiple bottlenecks: First, the generated images have obvious defects at the detail level, such as blurred traffic sign text, distorted vehicle outlines, and broken road markings. This is mainly due to the generator's lack of explicit constraints on the semantic structure of the scene. Second, existing methods generally neglect the consistent modeling of the imaging physical process, including inherent sensor properties such as lens distortion, camera response characteristics, and rolling shutter effects. As a result, although the generated images are visually close to natural images, they lack the physical credibility of real drone-collected images. Third, traditional generative models suffer from stability issues during training. For example, adversarial training is prone to mode collapse, while diffusion models, although generating high-quality images, suffer from huge computational overhead due to their iterative generation mechanism, making it difficult to meet the application requirements for real-time data generation in traffic monitoring scenarios.

[0004] The existing paper, "Synthesizing Realistic Traffic Events from UAV Perspectives: A Mask-guided Generative Approach Based on Style-modulated Transformer," proposes a UAV incident image synthesis scheme based on style-modulated Transformer, solving the problems of overall style transfer and basic semantic fidelity from simulation to UAV, and achieving improved visual realism while preserving the main structure. However, it still has the following shortcomings: reliance on hard masks, lack of uncertainty and geometric alignment; no modeling of camera physics (CRF / rolling shutter / noise), resulting in insufficient physical consistency; and limited discrimination and training steady-state / generalization capabilities. Summary of the Invention

[0005] The present invention provides a method and system for generating UAV accident images that integrates multimodal semantic constraints and camera physical perception.

[0006] The technical solution to achieve the purpose of this invention is: a method for generating UAV accident images that integrates multimodal semantic constraints and camera physical perception, comprising the following steps:

[0007] S1. Perform panoramic segmentation and open vocabulary alignment on the simulated traffic event image. Combine monocular depth and road geometry priors to generate a soft semantic mask and class prototype library with category uncertainty weights as joint spatial and category constraints.

[0008] S2. Construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, fuse soft mask statistics and class prototypes through semantic modulation across the attention normalization module, and then supplement with discrete wavelet frequency domain paths to inject high-frequency details.

[0009] S3. A camera response encoder is introduced to extract sensor style codes from real UAV images, including camera response function, rolling shutter and noise spectrum. A convolution operation that integrates whitening and coloring transformation and weighted modulation and demodulation mechanism is used to achieve controllable injection and stabilization of global style and sensor characteristics.

[0010] S4. Design a panoramic conditional multi-expert discriminator that integrates semantic embedding, geometric consistency constraints, and momentum memory to achieve consistent discrimination across scales and priors.

[0011] S5. A hybrid paradigm of hinged adversarial loss and dual ranking loss is adopted, and an exponential moving average (EMA) teacher network and a dual time scale update strategy are introduced to improve convergence stability and realism.

[0012] S6. Perform semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and optimize CVaR by dividing the bar to enhance out-of-domain generalization;

[0013] S7. Using a differentiable camera simulator, physical consistency style rendering of lens distortion, rolling shutter, and noise injection is performed based on sensor style codes to output traffic accident images with realistic UAV imaging characteristics.

[0014] A system for generating UAV accident images that integrates multimodal semantic constraints and camera physical awareness includes:

[0015] The semantic and geometric prior construction module is used to perform panoramic segmentation and open vocabulary alignment on simulated traffic event images. It combines monocular depth and road geometric priors to generate soft semantic masks and class prototype libraries with category uncertainty weights as joint spatial and category constraints.

[0016] The dual-branch generation network module is used to construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, it fuses soft mask statistics and class prototypes through semantic modulation across the attention normalization module SCAN, and then supplements it with discrete wavelet frequency domain paths to inject high-frequency details.

[0017] The sensor style injection module is used to introduce the sensor style code extracted from real UAV images by the camera response encoder, including the camera response function, rolling shutter and noise spectrum. It adopts a convolution operation that integrates whitening and coloring transformation as well as weight modulation and demodulation mechanism to achieve controllable injection and stabilization of global style and sensor characteristics.

[0018] The panoramic condition discrimination module is used to design a panoramic condition multi-expert discriminator, which integrates semantic embedding, geometric consistency constraints and momentum memory to achieve consistent discrimination across scales and priors;

[0019] The training strategy optimization module is used to adopt a hybrid paradigm of hinged adversarial loss and dual ranking loss, and introduces an exponential moving average (EMA) teacher network and a dual timescale update strategy to improve convergence stability and realism.

[0020] The generalization enhancement module is used for semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and CVaR optimization to enhance out-of-domain generalization.

[0021] The physical camera simulation module is used to perform physical consistency style rendering of lens distortion, rolling shutter and noise injection based on sensor style codes using a differentiable camera simulator, and output traffic accident images with realistic UAV imaging characteristics.

[0022] Compared with existing technologies, the significant advantages of this invention are: (1) This invention proposes a feature modulation mechanism based on multimodal semantic constraints, which fuses soft mask statistics and class prototype library through semantic regulation across the attention normalization module SCAN. Compared with existing semantically guided generation methods (such as SPADE), it effectively solves the problems of geometric distortion and detail blurring of small-scale targets such as traffic signs and vehicle outlines under the perspective of UAV overhead shots, and maintains the structural consistency of generated images; (2) This invention designs a hierarchical style-aware generation architecture, which realizes the separation and control of global style attributes and local content expression through deformable Transformer combined with sensor style codes. Compared with CNN-based generation models (such as UVCGAN), this invention effectively solves the problems of geometric distortion and detail blurring of small-scale targets such as traffic signs and vehicle outlines under the perspective of UAV overhead shots, and maintains the structural consistency of generated images; Compared with V2), it breaks through the limitation of local receptive field modeling of complex scenes, and can effectively decouple and transfer the illumination changes, texture features and camera physical characteristics in real UAV images. It shows excellent visual consistency under different weather conditions and multi-target interaction scenarios. (3) The present invention constructs an adversarial training framework with enhanced discrimination mechanism. By integrating multi-expert discriminators and momentum memory, and combining the cross-scale consistency constraint of semantic embedding, it significantly alleviates the mode collapse problem compared with the standard generative adversarial training method. While ensuring the generation quality, it supports the stable generation of high-resolution (640×640) images, meeting the dual requirements of real-time performance and reliability in practical applications. Attached Figure Description

[0023] Figure 1 Image synthesized using UVCGAN V2.

[0024] Figure 2 This is a flowchart illustrating the complete technical solution of the present invention.

[0025] Figure 3 This is a schematic diagram illustrating the semantic prior information extraction of the present invention.

[0026] Figure 4 This is a schematic diagram of the semantic-guided multi-scale feature modulation and enhancement process of the present invention.

[0027] Figure 5 This is a schematic diagram of the style transfer based on the physical model of the present invention.

[0028] Figure 6 This is a schematic diagram of the multi-prior joint discrimination mechanism of the present invention.

[0029] Figure 7 This is a schematic diagram of the multi-objective optimization system of the present invention.

[0030] Figure 8 This is a schematic diagram of the generalization enhancement framework of the present invention.

[0031] Figure 9 This is a schematic diagram of the physical camera simulation pipeline calculation of the present invention. Detailed Implementation

[0032] This invention relates to a UAV accident image generation method that integrates multimodal semantic constraints and camera physical perception. The method is based on a Transformer architecture that combines multimodal semantic constraints with camera-aware style modulation. Its main steps include: generating soft semantic masks and class prototype libraries through panoramic segmentation and open vocabulary alignment to achieve joint spatial and category constraints on traffic scenes; constructing an encoder-decoder network that combines content and global style, and utilizing a semantically modulated cross-attention normalization module to achieve deep information fusion; extracting sensor style codes and camera responses from real UAV images to achieve precise style and content injection and modulation; employing a panoramic conditional multi-expert discriminator to ensure the geometric and semantic consistency of the generated images, and introducing a hybrid strategy of adversarial loss and dual ranking loss to improve the quality and stability of the generated images; in the style implementation stage, injecting lens distortion, rolling shutter, and noise using a differentiable camera simulator to simulate the imaging characteristics of real UAVs, ultimately outputting realistic traffic accident images. This invention not only improves the performance of simulated images but also provides a new solution for the dynamic generation of complex traffic scenes, demonstrating significant application potential and research value.

[0033] Currently, drone-based traffic event image generation technology faces three core challenges: the scarcity of real data, the visual differences between simulation and real-world scenes, and the balance between detail preservation and efficiency in generative models. Due to the limited flight time of drones and the low frequency of accidents, it is difficult to obtain sufficient real-world data. While images generated by simulation platforms (such as CARLA) can compensate for the lack of data, they suffer from inter-domain differences such as texture distortion and unnatural lighting, leading to a significant decrease in the generalization ability of detection models trained on synthetic data in real-world scenarios. Among existing generative methods, CNN-based generative adversarial networks (such as UVCGAN V2) are prone to edge blurring or geometric distortion when reconstructing fine structures such as road markings and traffic signs. Figure 1 As shown, while the diffusion model can generate high-quality images, its iterative generation mechanism results in huge computational overhead, making it difficult to meet the real-time requirements of practical applications.

[0034] To address the aforementioned problems, this invention proposes a method for generating UAV accident images that integrates multimodal semantic constraints and camera physical perception. For example... Figure 2As shown, this method first extracts soft semantic masks and class prototype libraries with uncertain weights from simulated traffic images through panoramic segmentation and open vocabulary alignment techniques, providing joint spatial and category-level constraints for the generation process. Subsequently, a dual-branch generation architecture including a CNN encoder-decoder and a deformable Transformer is constructed. Semantic priors are fused at multiple scales through a semantic modulation cross-attention normalization (SCAN) module, and discrete wavelet transform is used to enhance the expressiveness of high-frequency details.

[0035] To enhance the physical realism of the generated images, this invention introduces a camera response encoder to extract sensor style codes (including CRF parameters, noise spectra, etc.) from real UAV images. Through convolutional operations that fuse whitening and coloring transformations with weighted modulation and demodulation mechanisms, controllable injection and stabilization of global style and sensor characteristics are achieved. In the discriminator design, a multi-expert discriminator structure is adopted, combining geometric consistency (horizon / road homography) and sensor characteristic constraints, and using a momentum memory to achieve cross-scale discrimination. During training, a hybrid paradigm of hinged adversarial loss and dual ranking loss, combined with an EMA teacher network and a dual-timescale update strategy, effectively improves training stability and generation quality.

[0036] To address the out-of-domain generalization problem, this invention enhances model robustness through semantic hierarchical masking reconstruction and a view-geometric perception block-level retrieval mechanism, and employs Conditional Value at Risk (CVaR) optimization criteria to focus on performance improvement in the worst-case scenarios. Finally, a differentiable physical camera simulation pipeline transforms sensor style codes into physical imaging characteristics such as lens distortion and rolling shutter effects, outputting UAV-view images that possess both semantic consistency and physical realism. This method, while maintaining generation efficiency (supporting real-time generation at 640×640 resolution), significantly improves the performance of generated images in terms of structural detail preservation and illumination consistency, providing a high-quality training data foundation for UAV traffic incident detection models.

[0037] The following is a detailed description of the specific steps of a UAV accident image generation method that integrates multimodal semantic constraints and camera physical perception according to the present invention.

[0038] like Figure 2 As shown, the UAV accident image generation method that integrates multimodal semantic constraints and camera physical perception includes the following steps:

[0039] S1. Perform panoramic segmentation and open vocabulary alignment on the simulated traffic event image. Combine monocular depth and road geometry priors to generate a soft semantic mask and class prototype library with category uncertainty weights as joint spatial and category constraints.

[0040] S11. By performing panoramic segmentation and open vocabulary alignment, the Grounded-SAM model is used to process the input simulated traffic event image. The process generates a set of semantic union masks containing N instances. For each instance Visual feature vectors are extracted using a pre-trained feature extractor. Extract predefined category sets using CLIP and OWL-ViT models. Each category Text features ,in Represents the total number of categories in a predefined category set; calculation example. With category semantic similarity matrix between Its elements are obtained by cosine similarity and normalization:

[0041]

[0042] in, express Norm; this formula ensures similarity. ;

[0043] S12, Based on the semantic similarity matrix Calculate each instance The category probability distribution and its uncertainty. For example... Figure 3 As shown, firstly, the smoothness of the probability distribution is adjusted using temperature scaling techniques:

[0044]

[0045] in For temperature parameters; when When the distribution becomes smoother, uncertainty increases; when At that time, the distribution becomes more acute and the uncertainty decreases.

[0046] Then, each instance is calculated. Uncertainty , defined as the difference between the probability and the highest confidence level:

[0047]

[0048] Define instance-level reliability weights. for:

[0049]

[0050] Finally, the instance-level weights and class probabilities are extended to the pixel level to generate a soft semantic mask. ,in Indicates the total number of categories. Semantic mask of high, Semantic mask Width; for instances pixel position In category The soft mask value is:

[0051]

[0052] In this way, soft semantic masking It effectively integrates the probability distribution of categories with the uncertainty measure of segmentation.

[0053] S13. Utilize the monocular depth map estimated from the image. road plane homography matrix soft semantic masking Perform global geometric alignment to conform to geometric constraints from a bird's-eye view; Each category channel Using homography matrix Perform perspective transformation The aligned mask is obtained by bilinear interpolation. :

[0054]

[0055] After performing the above operation on all categories of channels, the alignment mask is obtained. ;

[0056] Subsequently, the alignment mask was adjusted using a Conditional Random Field (CRF). The boundary region is refined to ensure consistency. This is achieved by minimizing the following CRF energy function. The optimized mask is obtained. :

[0057]

[0058] in, The set of all pixels. It is a set of adjacent pixel pairs. To balance hyperparameters. Data items. Defined as the negative log-likelihood of pixel color and mask prediction, smoothing term Constraints such as the Potts model that encourage consistent labeling among adjacent pixels are employed. Finally, the solution is obtained using the Graph Cut algorithm.

[0059]

[0060] Obtain a geometrically consistent and boundary-precise soft semantic mask. .

[0061] S14. Based on the obtained soft semantic mask and its corresponding original image Build a learnable class prototype library ,in Indicates the total number of predefined categories. This represents the dimension of the feature vector; for an instance Its characteristics Through its mask area With images The calculation yielded:

[0062]

[0063] in This represents a feature extractor network based on momentum updates, used to extract features from an image. In the mask Depth visual features within a specified area;

[0064] Subsequently, those belonging to the same category All instance features Momentum aggregation is performed to form the prototype vector of this class. :

[0065]

[0066] in, For time steps, The momentum coefficient, It belongs to the category in the current batch. A collection of instances. Ultimately, the prototype vectors for all categories. Converging into a class prototype library This is used to perform category-level semantic modulation on the generation process in subsequent steps.

[0067] S2. Construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, fuse soft mask statistics and class prototypes through semantic modulation across the attention normalization module SCAN, and then supplement with discrete wavelet frequency domain paths to inject high-frequency details.

[0068] S21, Input Image 3 represents the RGB color channel. and The height and width of the input image are respectively, and it first passes through a preprocessing layer, such as... Figure 4As shown above, the initial feature map is obtained by using a 3×3 convolutional layer and the GeLU activation function. ,in This represents the number of channels in the initial feature map; subsequently, It is fed into a five-stage encoder-decoder structured as a U-Net. Let the encoder's... class The output feature map is Decoder number The feature map corresponding to level is ,in Indicates the first The number of channels in the level feature map. and They represent the first The height and width of the level feature map; features are fused between encoders and decoders via skip connections. At each level... All embed semantic modulation cross-attention normalization SCAN mechanism to affect features and Modulation is performed.

[0069] S22, at each level Perform semantic modulation across attention normalization SCAN to obtain a refined soft semantic mask. and class prototype library Integrating into the current feature as a semantic condition .

[0070] First, analyze the current feature map. Perform layer normalization:

[0071]

[0072] in and It is a feature map The mean and standard deviation, To prevent division by zero for extremely small constants.

[0073] Subsequently, modulation parameters are generated using a cross-attention mechanism. and The class prototype library As the key K and value V, the normalized features With soft mask The fusion representation is used as query Q:

[0074]

[0075]

[0076]

[0077]

[0078] in, , , These represent the learnable linear projection layers used to generate queries, keys, and values, respectively. This represents a learnable linear projection layer used to generate modulation parameters; This represents the normalized exponential function; Represents the key vector The feature dimension is used as a scaling factor; This represents the matrix transpose operation. For adaptive pooling operations, downsampling to Same space size ; For adaptive instance normalization, the pooled mask statistics are used for features. Modulation is performed to initially fuse semantic information;

[0079] Next, the generated parameters are used to modulate the normalized features:

[0080]

[0081] in This indicates element-wise multiplication. Modulated characteristics. It retains the original content information while injecting global category constraints from semantic masks and class prototypes.

[0082] S23. Construct a global style branch, the core of which is a deformable self-attention Transformer network called Deformable ViT. This branch is as follows: Figure 4 As shown below, deep features of the encoder As input, a learnable style dictionary is introduced at each layer. With a style router implemented using lightweight networking. ,in This indicates the total number of style prototypes contained in the style dictionary. This represents the feature dimension of each style prototype, adaptively selecting the hierarchical style code that matches the content of the current scene. :

[0083]

[0084] Selected style code Global style modulation for subsequent steps.

[0085] S24. Features modulated by the SCAN module Perform multi-scale frequency domain enhancement. Perform a two-dimensional discrete wavelet transform (DWT) to decompose it into four parallel subbands: the low-frequency subband. and high-frequency detail subbands representing the horizontal direction , representing the high-frequency detail subband in the vertical direction , representing the high-frequency detail subband in the diagonal direction :

[0086]

[0087] For high-frequency subband , , use Convolution is performed with lightweight modulation to enhance detail. Subsequently, the processed subbands are subjected to Inverse Discrete Wavelet Transform (IDWT) and fused back into the feature space to obtain the final output features with enhanced high-frequency details. :

[0088]

[0089] This feature This will be used as the output of the current scale and applied to the next scale or final image generation.

[0090] S3. A camera response encoder is introduced to extract sensor style codes from real UAV images, including camera response function, rolling shutter and noise spectrum. A convolution operation that integrates whitening and staining transformation and weighted modulation and demodulation mechanism is adopted to achieve controllable injection and stabilization of global style and sensor characteristics.

[0091] S31. Construct a camera response encoder from real UAV images. Extracting sensor style codes The encoder consists of a pre-trained CNN backbone network and a multilayer perceptron (MLP), and its output is a parameterized vector:

[0092]

[0093] The sensor style code It is a multidimensional vector whose dimensions encode physical imaging characteristics such as the parameters of the camera response function (CRF), the color matrix, the approximate value of ISO / gain, the rolling shutter readout time parameter, and the noise power spectral density.

[0094] S32. Receive the final output feature map after wavelet frequency domain enhancement. ,Right now and utilize hierarchical style codes With sensor style code Style adjustment.

[0095] First, the sensor style code With hierarchical style codes Fusion, generating the statistical mean of the target style. Covariance Matrix :

[0096]

[0097] in This represents a vector concatenation operation. This represents a learnable linear mapping layer used to map the concatenated style vectors to the mean and covariance statistics of the target style, such as... Figure 4 As shown in the dashed box.

[0098] Simultaneously, calculate content features Statistic and Subsequently, a whitening and coloring transformation (WCT) is performed to map the statistical properties of the content features to the target style statistical space:

[0099]

[0100] in This represents an intermediate feature map that exhibits the statistical characteristics of the target style after whitening staining transformation;

[0101] To ensure the stability of generated features and prevent amplitude distortion caused by style injection, orthogonal convolution is introduced. Demodulation with RMS:

[0102]

[0103] in, This means that its weight matrix satisfies the orthogonality constraint. Convolutional layers are used to stabilize feature distribution; The root mean square normalization is calculated as follows: This is used to constrain the characteristic amplitude and ensure the numerical stability of the results.

[0104] S33, Modulated features Constraints are applied in the frequency domain to prevent excessive stylization from causing color distortion and contrast shift. The specific process is as follows: Figure 5 As shown at the bottom. (Regarding...) Perform a short-time Fourier transform (STFT) and apply bandwidth limiting and energy clamping in the frequency domain to obtain the frequency domain characteristic signal after frequency domain constraint processing. :

[0105]

[0106] in, Operational retention frequency radius Low-frequency components within this range The operation limits the frequency domain energy to a threshold. Within. Finally, the processed frequency domain signal is transformed using the Inverse Short Time Fourier Transform (ISTFT). Transforming back to the spatial domain yields the final modulated features. :

[0107]

[0108] After the above steps, the features It combines the sensor style of real cameras with the stability of content, laying the foundation for generating physically realistic images.

[0109] S4. Design a "panoramic condition multi-expert discriminator" that integrates semantic embedding and geometric consistency constraints (horizon / vanishing point, road homography, shadow direction) and momentum memory to achieve consistent discrimination across scales and priors.

[0110] S41. Construct a system composed of Expert discriminant A collection of components. Each expert Focusing on different scales or different semantic category subspaces of an image. For example... Figure 6 As shown, each expert discriminator uses the obtained semantic embedding As a condition, prior information is injected through class-conditional normalization, thus associating its discriminative ability with the semantic content of the image. This process is represented as:

[0111]

[0112] in, The image to be judged, Features of the intermediate layer of the discriminator For the conditional normalization layer, its statistics and From the condition vector It was calculated through a small mapping network, and with the help of different experts It changes dynamically depending on the category.

[0113] S42. From the image to be discriminated Estimating geometric properties, including vanishing point ,horizon and road homography matrix Calculate these geometric properties and their relation to semantic road regions in the image. Consistency score :

[0114]

[0115] in, , , It is based on semantic road mask The calculated ground truth value of the true geometric attributes; this score measures how well the generated image matches the semantic prior in terms of geometric structure.

[0116] S43, Predicted Image Sensor-related properties, including rolling shutter stripe intensity noise spectrum Reprojection error of camera response function (CRF) Match these attributes with the style codes of real sensors. The corresponding parameters are compared to obtain the sensor consistency score. :

[0117]

[0118] This score measures how well the generated image matches the target's real-world camera style in terms of its underlying physical imaging characteristics;

[0119] S44. Employ a momentum contrast-based mechanism to maintain a dynamically updated global feature library. This feature library is used to store multi-scale features of real and generated images. It is maintained using a momentum update strategy.

[0120]

[0121] in, and These represent the current time step. and the previous time step The global feature library status, Features extracted from real and generated images for the current batch. This is the momentum coefficient; this feature library is used for hard sample mining and cross-domain feature matching to enhance the discriminant's ability.

[0122] S45, Final discriminator output It is a weighted sum of the expert judgment results, geometric consistency score, and sensor consistency score, achieving multi-dimensional and multi-prior joint judgment:

[0123]

[0124] in, The weights for each expert discriminator can be learned or fixed. and This is a hyperparameter used to balance the strength of the adversarial loss and the geometric and sensor consistency constraints.

[0125] S5. A hybrid paradigm of hinged adversarial loss and dual ranking loss is adopted, and an exponential moving average (EMA) teacher network and a dual time scale update strategy are introduced to improve convergence stability and realism.

[0126] S51. The overall training objective of the method consists of adversarial loss, ranking loss, and multiple consistency loss. Discriminator The parameters are obtained by minimizing hinge loss. Update:

[0127]

[0128] in, It is the discriminator's recognition of the real image. The output, It is the discriminator that determines the generated image The output, Indicates the distribution of real images The expectation of upsampling, Indicates the noise prior distribution The expectation of upsampling, Represents a generator;

[0129] like Figure 7 As shown by the dashed line, the generator The parameters combat loss by minimizing the hinge. With dual sorting loss Update:

[0130]

[0131]

[0132]

[0133] in, Indicates the distribution of prior noise Two independently sampled random noise vectors and Find the expected value. It is a balancing hyperparameter. For marginal parameters; at the same time, for the discriminator Spectral normalization is applied to all layers to stabilize the training process;

[0134] S52. Maintain a teacher generator using the Exponential Moving Average (EMA) strategy. After each training iteration, the student generator is... That is, the parameters of the currently trained master generator. Perform momentum update to obtain the parameters of the teacher generator. :

[0135]

[0136] in, This is the momentum decay coefficient. During the inference phase, a teacher generator is used. Perform image generation to obtain more stable and higher quality output.

[0137] The generator employs the dual-timescale update rule TTUR. and discriminator Set different learning rates to further balance the training progress of both:

[0138]

[0139] in, and These represent the learning rates of the generator and discriminator, respectively. Based on the learning rate, This is a proportional hyperparameter;

[0140] S53. To strictly constrain the quality of the generated images, several consistency loss functions based on prior knowledge are introduced. First, the calculated geometric consistency score is used... Constructing geometric consistency loss To maximize the consistency of the generated image with the real image in terms of geometric properties:

[0141]

[0142] Secondly, calculate the soft semantic mask obtained by processing the generated image. With input noise Corresponding initial semantic conditions The differences between them are used to construct semantic consistency loss. :

[0143]

[0144] Next, the calculated sensor consistency score is used. Constructing sensor consistency loss To maximize the consistency between the generated image and the target camera style:

[0145]

[0146] Then, the CLIP model is used to calculate and generate the image. With input text description Cosine similarity as text-image alignment loss :

[0147]

[0148] in, and These are the image encoder and text encoder of the CLIP model, respectively.

[0149] Finally, the generator The total loss function is:

[0150]

[0151] in, The weighting hyperparameters are used to assign weights to each loss.

[0152] S6. Perform semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and optimize CVaR by dividing the bar to enhance out-of-domain generalization;

[0153] S61. During training, such as Figure 8 As shown in the dashed box on the left, the deep feature map from the encoder... Hierarchical block masking is implemented to enhance the model's ability to reconstruct parts of the observation data and improve its generalization ability. , , These represent the number of channels, height, and width of the deep feature map, respectively. The masking strategy is based on soft semantic masking. The indicated category information is adaptively adjusted. Specifically, the feature map is divided into... The blocks, each block masking probability By its dominant category Rarity weight Decide:

[0154]

[0155] in, It is a basic masking rate hyperparameter. It is a weighting factor that is inversely proportional to the frequency of occurrence of the category. For rare categories such as traffic signs, For common categories such as roads, Finally, a binary mask is generated. ,in Representation block The mask is used to discard corresponding feature blocks during training, forcing the generator to learn more robust feature representations.

[0156] S62. To enhance the geometric rationality of the generated content, such as Figure 8 As shown in the dashed box in the middle, cross-view block retrieval and fusion are performed in the BEV space of the bird's-eye view. This is achieved using the estimated road plane homography matrix. The feature blocks of the generated image Projected into BEV space, and compared with BEV feature blocks in a real UAV image database. Perform similarity matching. Cosine similarity is used for similarity calculation:

[0157]

[0158] in, It is a feature extraction function. If the similarity exceeds a threshold... If the match is successful, then it is considered a successful match.

[0159]

[0160] in, This indicates the block matching result. This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. For a successfully matched block, its features are weighted and fused with the nearest neighbor features found in the database. Then, the fused features are subjected to inverse homography transformation. Projecting back to the original viewpoint and replacing the original features injects geometric and appearance priors from real data.

[0161] S63. Adopt the Conditional Value at Risk (CVaR) optimization criterion, such as... Figure 8 As shown in the dashed box on the right, in the total loss function Based on this, the model's performance under the worst environmental conditions is optimized, thereby greatly improving its out-of-domain generalization ability. Given a confidence level... The CVaR optimization problem is defined as:

[0162]

[0163] in, It is the defined total loss, which includes adversarial loss, ordering loss, and multiple consistency losses. It is a random variable digits at the tail of the distribution The expected value of the above conditions is approximately equal to the worst-case scenario. The average loss under various conditions. By minimizing CVaR, the model is guided to focus on optimizing performance under "hard samples" or "rare environments," thereby significantly enhancing the model's robustness.

[0164] S7. Using a differentiable camera simulator, physical consistency style rendering of lens distortion, rolling shutter, and noise injection is performed based on sensor style codes to output traffic accident images with realistic UAV imaging characteristics.

[0165] S71. Construct a differentiable physical camera simulation pipeline. The specific process is as follows: Figure 9 As shown, the pipeline generates preliminary images and extracts sensor style codes. Using this as input, a series of parametric physical transformations are used to generate a final image with a highly realistic style. .

[0166] The simulation pipeline performs the following micromanipulations in the physical sequence of real camera imaging: First, lens distortion simulation is performed, using... distortion coefficients of the encoding ,in , Represents the radial distortion coefficient. , Represents the tangential distortion coefficients, which affect the generated image. pixel coordinates A distortion transformation is performed, and the distorted image is obtained through differentiable bilinear sampling. Subsequently, a rolling shutter simulation was performed, using... Line readout time of the Chinese encoding And camera motion parameters, for the distorted image Apply line-by-line offset to simulate the rolling shutter effect:

[0167]

[0168] in This represents the output image after simulating the rolling shutter effect. and These represent the row index and column index of the pixel, respectively, and the row offset. It is a line number A function of camera movement speed;

[0169] Finally, color response and noise injection are performed using... Camera Response Function (CRF) parameters and color matrix encoded in Chinese For images Perform color transformation, and according to The noise model encoded in the code generates the corresponding noise map. Add to the image:

[0170]

[0171] in, This represents the final image after noise injection. This represents the image after color response transformation. This represents the generated additive noise map;

[0172] S72. If the input real UAV image includes EXIF ​​metadata or precise camera calibration information Then, this information is used to determine the sensor style code. Online fine-tuning is performed to further improve the accuracy of the physical simulation. This is achieved by establishing a weakly supervised calibration loss function:

[0173]

[0174] in, It is an encoder that maps EXIF ​​information to style code space. This loss is added to the total loss. In this process, it is optimized together with other objectives of the generator to achieve end-to-end sensor style code fine-tuning based on real camera data.

[0175] After processing by the differentiable physics simulation pipeline described above, the final output image retains both the generated semantics, content, and global style, as well as the style of the extracted real physical sensors. The image is visually highly consistent with real traffic accident images captured by UAVs and can be used for downstream tasks such as data augmentation and simulation testing.

[0176] Training uses joint loss:

[0177]

[0178] The preferred settings for each weight are as follows: , , , , , .

[0179] The training data includes simulated source domain and real UAV target domain images. The target domain preferably includes samples from multiple UAV datasets (including but not limited to UAVDT, AU-AIR, VDD, UAVid, VisDrone) and self-sampled images. Subdomain equalization is performed based on weather, altitude, and viewpoint coverage. The implementation memory size is preferably 16k per domain, with a momentum coefficient of 0.999 and a certain number of experts. .

[0180] Based on the same inventive concept, the present invention also provides a UAV accident image generation system that integrates multimodal semantic constraints and camera physical perception, comprising:

[0181] The semantic and geometric prior construction module is used to perform panoramic segmentation and open vocabulary alignment on simulated traffic event images. It combines monocular depth and road geometric priors to generate soft semantic masks and class prototype libraries with category uncertainty weights as joint spatial and category constraints.

[0182] The dual-branch generation network module is used to construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, it fuses soft mask statistics and class prototypes through semantic modulation across the attention normalization module SCAN, and then supplements it with discrete wavelet frequency domain paths to inject high-frequency details.

[0183] The sensor style injection module is used to introduce the sensor style code extracted from real UAV images by the camera response encoder, including the camera response function, rolling shutter and noise spectrum. It adopts a convolution operation that integrates whitening and coloring transformation as well as weight modulation and demodulation mechanism to achieve controllable injection and stabilization of global style and sensor characteristics.

[0184] The panoramic condition discrimination module is used to design a panoramic condition multi-expert discriminator, which integrates semantic embedding, geometric consistency constraints and momentum memory to achieve consistent discrimination across scales and priors;

[0185] The training strategy optimization module is used to adopt a hybrid paradigm of hinged adversarial loss and dual ranking loss, and introduces an exponential moving average (EMA) teacher network and a dual timescale update strategy to improve convergence stability and realism.

[0186] The generalization enhancement module is used for semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and CVaR optimization to enhance out-of-domain generalization.

[0187] The physical camera simulation module is used to perform physical consistency style rendering of lens distortion, rolling shutter and noise injection based on sensor style codes using a differentiable camera simulator, and output traffic accident images with realistic UAV imaging characteristics.

[0188] The specific implementation methods of the above modules are the same as the steps of the aforementioned UAV accident image generation method, and will not be repeated here.

[0189] This invention innovatively proposes a UAV accident image generation method that integrates multimodal semantic constraints and camera physical perception. By fusing panoramic segmentation semantic priors, physical sensor characteristic modeling, and frequency domain enhancement mechanisms, it systematically solves the challenge of coordinating visual realism, geometric consistency, and physical accuracy in traffic event generation. The method first utilizes Grounded-SAM and CLIP open vocabulary models to extract soft semantic masks and class prototype libraries with uncertain weights from simulated data. Then, a spatial attention denormalization module (SCAN) dynamically injects semantic statistics into the multi-scale feature encoding-decoding process, significantly improving the structural fidelity of small objects such as traffic signs and road markings. Furthermore, it innovatively designs a deformable Transformer style branch and a camera response encoder to extract sensor style codes containing CRF parameters, rolling shutter characteristics, and noise spectra from real UAV images. Controllable injection of physical characteristics is achieved through whitening-coloring transform (WCT) and orthogonal constraint convolution.

[0190] This invention proposes a panoramic conditional multi-expert discriminator architecture, integrating geometric consistency constraints (vanishing point / road homography) and a momentum memory mechanism. It enhances the stability of adversarial training through cross-scale semantic embedding and adaptive block retrieval strategies. A hybrid paradigm of hinge adversarial loss and dual ranking loss is employed, combined with an EMA teacher network and dual-timescale updates, to reduce the FID index while maintaining real-time generation at a high resolution of 640×640. Finally, a differentiable camera simulation pipeline achieves physically accurate modeling of lens distortion, rolling shutter speed, and noise, outputting UAV-view images with semantic consistency, visual realism, and sensor characteristics, providing a training data foundation for intelligent transportation systems that surpasses traditional simulation methods.

[0191] Example

[0192] This embodiment uses the following data and environment settings: the dataset consists of 7,638 simulated GTA V platform images and 3,200 real drone images; the hardware and software environment consists of two NVIDIA A800 (80GB) graphics cards, one Intel Platinum Xeon 8358 @ 2.60GHz processor, Ubuntu 20.04 operating system, Python 3.8, PyTorch 1.11.0, and CUDA 11.3. The training and evaluation protocols are consistent with the control method, and the evaluation metrics are FID and KID (both using Inception-V3 2048-dimensional features, data size aligned), the smaller the value, the better the generation effect. Table 1 shows the comparison results of the generation quality of the present invention and the existing method under the same evaluation protocol.

[0193] Table 1. Comparison of the quality generated by the present invention and existing methods.

[0194]

[0195] The table includes the literature titled "Synthesizing Realistic Traffic Events from UAVPerspectives: A Mask-guided Generative Approach Based on Style-modulated Transformer".

[0196] As shown in Table 1, under consistent data and evaluation settings, the method of this invention achieves further reductions in both core metrics: compared to existing literature (No. 9), FID decreases from 73.44 to 68.12, a reduction of approximately 7.24%, and KID decreases from 3.68 to 3.21, a reduction of approximately 12.77%; compared to the current best Transformer baseline (No. 8), FID decreases by approximately 14.66%, and KID decreases by approximately 16.62%; compared to the diffusion model baseline (No. 6), FID and KID decrease by approximately 16.49% and 18.94%, respectively. These improvements are reflected in feature statistics that better reflect the distribution of real UAV data, fewer semantic mismatches and structural distortions, and more consistent imaging physical properties. Combining qualitative and quantitative evidence, the method of this invention demonstrates stronger overall advantages in terms of realism, detail preservation, geometric consistency, and sensor consistency.

[0197] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of protection of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for UAV accident image generation by fusing multi-modal semantic constraints with camera physical perception, characterized in that, Includes the following steps: S1. Perform panoramic segmentation and open vocabulary alignment on the simulated traffic event image. Combine monocular depth and road geometry priors to generate a soft semantic mask and class prototype library with category uncertainty weights as joint spatial and category constraints. S2. Construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, fuse soft mask statistics and class prototypes through semantic modulation across the attention normalization module, and then supplement with discrete wavelet frequency domain paths to inject high-frequency details. S3. A camera response encoder is introduced to extract sensor style codes from real UAV images, including camera response function, rolling shutter and noise spectrum. A convolution operation that integrates whitening and coloring transformation and weighted modulation and demodulation mechanism is used to achieve controllable injection and stabilization of global style and sensor characteristics. S4. Design a panoramic conditional multi-expert discriminator that integrates semantic embedding, geometric consistency constraints, and momentum memory to achieve consistent discrimination across scales and priors. S5. A hybrid paradigm of hinged adversarial loss and dual ranking loss is adopted, and an exponential moving average (EMA) teacher network and a dual time scale update strategy are introduced. S6. Perform semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and optimize CVaR by dividing the bar to enhance out-of-domain generalization; S7. Using a differentiable camera simulator, physical consistency style rendering of lens distortion, rolling shutter, and noise injection is performed based on sensor style codes to output traffic accident images with realistic UAV imaging characteristics. 2.The method of claim 1, wherein, Step S1 specifically includes: S11. By performing panoramic segmentation and open vocabulary alignment, the Grounded-SAM model is used to process the input simulated traffic event image. The process generates a set of semantic union masks containing N instances. For each instance Visual feature vectors are extracted using a pre-trained feature extractor. Extract predefined category sets using CLIP and OWL-ViT models. Each category Text features ,in Represents the total number of categories in a predefined category set; calculation example. With category semantic similarity matrix between Its elements are obtained by cosine similarity and normalization: ； wherein denotes norm; similarity is ensured by this formula ; S12、based on the semantic similarity matrix , calculate the class probability distribution of each instance and its uncertainty; first, adjust the smoothness of the probability distribution by using temperature scaling technology: ； wherein T is a temperature parameter; Subsequently, the uncertainty of each instance is calculated , defined as the difference in probability from the highest confidence. ； and defining the reliability weight of the instance level is: ； Finally, the instance-level weights and class probabilities are extended to the pixel level to generate a soft semantic mask. , Represents soft semantic mask of high, Represents soft semantic mask Width; for instances pixel position In category The soft mask value is: ； S13. Utilize the monocular depth map estimated from the image. road plane homography matrix soft semantic masking Perform global geometric alignment to conform to geometric constraints from a bird's-eye view; Each category channel Using homography matrix Perform perspective transformation The aligned mask is obtained by bilinear interpolation. : ； After performing the above operation on all classes of channels, the alignment mask is obtained ; Subsequently, a conditional random field (CRF) is used to refine the consistency of the boundary regions of the alignment mask ; and an optimized mask is obtained by minimizing the following CRF energy function :​ ； where, is a set of all pixels, is a set of neighboring pixel pairs, is a balancing hyper-parameter; data item is defined as the negative log-likelihood of the pixel color given the mask prediction, a smoothing term Potts model is adopted to encourage the constraint that neighboring pixel labels are consistent; finally, it is solved by a Graph Cut algorithm. ； Obtaining geometrically consistent and boundary precise soft semantic masks ; S14, according to the obtained soft semantic mask and its corresponding original image , construct a library of learnable class prototypes , denotes the dimension of the feature vector; for an instance , its features through its mask region with the image is calculated: ； wherein a feature extractor network representing a momentum update for extracting images in the mask a depth visual feature within the specified region; Subsequently, momentum is aggregated for all instance features belonging to the same category forming a prototype vector for the category :​ ； wherein, is the time step, is the momentum coefficient, is the set of instances in the current batch belonging to class ; finally, the prototype vectors of all classes converge into a class prototype library . 3.The method of claim 2, wherein, Step S2 specifically includes: S21, input image where 3 represents the RGB color channel, and respectively the height and width of the input image, first pass through a pre-processing layer, including a 3x3 convolution layer and GeLU activation function, to obtain the initial feature map where represents the number of channels of the initial feature map; then, is sent into a five-level encoder-decoder with U-Net structure; let the feature map output by the first level of the encoder be , and the feature map corresponding to the first level of the decoder be where represents the number of channels of the first level feature map, and respectively represent the height and width of the first level feature map; the features between the encoder and decoder are fused through the skip connection; at each level , the SCAN mechanism is embedded to modulate the features and ; S22、at each level performing semantic conditioning across attentional normalization SCAN, obtaining refined soft semantic mask and class prototype library as semantic conditions into current features ; First, the current feature map is layer normalized: ； wherein and is the mean and standard deviation of the feature map is a very small constant;​ Subsequently, modulation parameters are generated using a cross-attention mechanism and ; a library of class prototypes is used as keys and values , the fusion representation of normalized features and soft masks is used as a query : ； ； ； ； wherein, , , respectively represent learnable linear projection layers for generating queries, keys and values; represent a learnable linear projection layer for generating modulation parameters; represents a normalized exponential function; represents a key vector with a feature dimension used as a scaling factor; represents a matrix transpose operation, is an adaptive pooling operation that down-samples to the same spatial size as ; is an adaptive instance normalization that modulates the feature with the pooled mask statistics to preliminarily fuse semantic information; Next, the generated parameters are used to modulate the normalized features: ； wherein denotes element-wise multiplication; modulated features both preserve the original content information and inject global class constraints from the semantic mask and class prototypes; S23, construct a global style branch, the core of which is a deformable self-attention Transformer network Deformable ViT; the branch takes the encoded deep features of the input image as input, and introduces a learnable style dictionary at each layer and a style router implemented by a lightweight network , wherein represents the total number of style prototypes contained in the style dictionary, represents the feature dimension of each style prototype, so as to adaptively select the hierarchical style code matching the current scene content : ； selected style code global style modulation for subsequent steps S24, the features modulated by the SCAN module performing multi-scale frequency domain enhancement; on performing a two-dimensional discrete wavelet transform (DWT) to decompose it into four parallel sub-bands: a low-frequency sub-band and a high-frequency detail sub-band representing the horizontal direction a high-frequency detail sub-band representing the vertical direction a high-frequency detail sub-band representing the diagonal direction : ； On high frequency subband , , Using Convolution for light modulation to enhance the detail expressiveness; then, inverse discrete wavelet transform (IDWT) is performed on the processed subband to fuse back to the feature space to obtain the final output feature with enhanced high frequency details : ； This feature The output will be as the current scale and used for the next scale or final image generation.

4. The method of claim 3, wherein, Step S3 specifically includes: S31, constructing a camera response encoder from real UAV images to extract sensor style codes ; the encoder is composed of a pre-trained CNN backbone network and a multi-layer perceptron, MLP, whose output is a parametric vector: ； The sensor style code is a multi-dimensional vector whose dimensions encode the parameters of the camera response function CRF, the color matrix, the approximated values of ISO / gain, the rolling shutter readout time parameters and the noise power spectral density; S32, receive the final output feature map enhanced in the wavelet frequency domain i.e. and utilize the hierarchical style code and the sensor style code for style modulation; First, the sensor style code is fused with the hierarchical style code to generate the statistical mean and covariance matrix of the target style and : ； wherein denotes a vector concatenation operation, denotes a learnable linear mapping layer to map the concatenated style vector to the mean and covariance statistics of the target style; At the same time, statistics of content features and are calculated ; subsequently, a whitening and colorization transform (WCT) is performed, mapping the statistical properties of the content features to a target style statistical space: ； wherein denotes the intermediate feature map having the target style statistics after the whitening and coloring transform; After obtaining the orthogonal constraint convolution is introduced with the root mean square RMS demodulation operation: ； wherein, representing that its weight matrix satisfies orthogonality constraint convolutional layer, for stabilizing feature distribution; representing root mean square normalization, calculated as for constraining feature amplitude, ensuring numerical stability of results; S33, the modulated features Constraints are imposed in the frequency domain to prevent excessive stylization from causing color distortion and contrast drift; the Short-time Fourier transform (STFT) is performed, and bandwidth limiting and energy clipping are performed in the frequency domain to obtain a frequency-domain feature signal after frequency-domain constraint processing : ； in, Operational retention frequency radius Low-frequency components within this range The operation limits the frequency domain energy to a threshold. Within; the processed frequency domain signal is transformed using the Inverse Short Time Fourier Transform (ISTFT). Transforming back to the spatial domain yields the final modulated features. : 。 5. The method for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception according to claim 4, characterized in that, Step S4 specifically includes: S41. Construct a system composed of Expert discriminant The set of components; each expert Focusing on different scales or different semantic category subspaces of the image; each expert discriminator uses the resulting semantic embedding As a condition, prior information is injected through class-conditional normalization to correlate its discriminative ability with the semantic content of the image; this process is represented as: ； in, The image to be judged, Features of the intermediate layer of the discriminator For the conditional normalization layer, its statistics and From the condition vector It was calculated through a small mapping network, and with the help of different experts And it changes dynamically according to different categories; S42. From the image to be discriminated Estimating geometric properties, including vanishing point ,horizon and road homography matrix ; Calculate these geometric properties and semantic road regions in the image Consistency score : ； in, , , It is based on semantic road mask The calculated ground truth value of the true geometric attributes; this score measures how well the generated image matches the semantic prior in terms of geometric structure. S43, Predicted Image Sensor-related properties, including rolling shutter stripe intensity noise spectrum Reprojection error of camera response function (CRF) ; Match these attributes with real sensor style codes The corresponding parameters are compared to obtain the sensor consistency score. : ； This score measures how well the generated image matches the target's real-world camera style in terms of its underlying physical imaging characteristics; S44. Employ a momentum contrast-based mechanism to maintain a dynamically updated global feature library. This feature library is used to store multi-scale features of real and generated images; it is maintained using a momentum update strategy. ； in, and These represent the current time step. and the previous time step The global feature library status, Features extracted from real and generated images for the current batch. This is the momentum coefficient; this feature library is used for hard sample mining and cross-domain feature matching to enhance the discriminant's ability. S45, Final discriminator output It is a weighted sum of the expert judgment results, geometric consistency score, and sensor consistency score, achieving multi-dimensional and multi-prior joint judgment: ； in, Assigning weights to each expert discriminator. and This is a hyperparameter used to balance the strength of the adversarial loss and the geometric and sensor consistency constraints.

6. The method for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception according to claim 5, characterized in that, Step S5 specifically includes: S51. The overall training objective of the method consists of adversarial loss, ranking loss, and multi-consistency loss; discriminator. The parameters are obtained by minimizing the discriminator loss. Update: ； in, It is the discriminator's judgment of the real image The output, The discriminator analyzes the generated image. The output, Indicates the distribution of real images The expectation of upsampling, Indicates the noise prior distribution The expectation of upsampling, Represents a generator; generator The parameters combat loss by minimizing the hinge. With dual sorting loss Update: ； ； ； in, Indicates the distribution of prior noise Two independently sampled random noise vectors and Find the expected value. It is a balancing hyperparameter. For marginal parameters; at the same time, for the discriminator Spectral normalization is applied to all layers to stabilize the training process; S52. Maintain a teacher generator using the Exponential Moving Average (EMA) strategy. After each training iteration, the student generator is... That is, the parameters of the current training master generator. Perform momentum update to obtain the parameters of the teacher generator. : ； in, The momentum decay coefficient; during the inference phase, a teacher generator is used. Perform image generation; The generator employs the dual-timescale update rule TTUR. and discriminator Set different learning rates to further balance the training progress of both: ； in, and These represent the learning rates of the generator and discriminator, respectively. Based on the learning rate, This is a proportional hyperparameter; S53. Introduce multiple consistency loss functions based on prior knowledge; first, utilize the calculated geometric consistency score... Constructing geometric consistency loss To maximize the consistency of the generated image with the real image in terms of geometric properties: ； Secondly, calculate the soft semantic mask obtained by processing the generated image. With input noise Corresponding initial semantic conditions The differences between them are used to construct semantic consistency loss. : ； Next, the calculated sensor consistency score is used. Constructing sensor consistency loss To maximize the consistency between the generated image and the target camera style: ； Then, the CLIP model is used to calculate and generate the image. With input text description Cosine similarity as text-image alignment loss : ； in, and These are the CLIP model's image encoder and text encoder, respectively. Finally, the generator The total loss function is: ； in, The weighting hyperparameters are used to assign weights to various losses.

7. The method for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception according to claim 6, characterized in that, Step S6 specifically includes: S61. During training, the deep feature maps from the encoder are processed... Implement layered block masking, in which , , These represent the number of channels, height, and width of the deep feature map, respectively; the masking strategy is based on soft semantic masks. The indicated category information is adaptively adjusted; the feature map is divided into The blocks, each block masking probability By its dominant category Rarity weight Decide: ； in, It is a basic masking rate hyperparameter. It is a weighting factor that is inversely proportional to the frequency of occurrence of a category; ultimately, a binary mask is generated. ,in Representation block Hidden; S62. Perform cross-view block retrieval and fusion in the bird's-eye view BEV3 space; utilize the estimated road plane homography matrix. The feature blocks of the generated image Projected into BEV space, and compared with BEV feature blocks in a real UAV image database. Similarity matching is performed; cosine similarity is used for similarity calculation. ； in, It is a feature extraction function; if the similarity exceeds a threshold If the match is successful, then it is considered a successful match. ； in, This indicates the block matching result. This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. For a successfully matched block, its features are weighted and fused with the nearest neighbor features found in the database. Then, the fused features are subjected to inverse homography transformation. Projecting back to the original viewpoint and replacing the original features, thereby injecting geometric and appearance priors from real data; S63. Using the Conditional Value at Risk (CVaR) optimization criterion, in the total loss function... Based on this, optimize the model's performance under the worst environmental conditions; given a confidence level The CVaR optimization problem is defined as follows: ； in, It is the defined total loss, which includes adversarial loss, ordering loss, and multiple consistency losses; It is a random variable digits at the tail of the distribution The above conditions are expected.

8. The method for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception according to claim 7, characterized in that, Step S7 specifically includes: S71. Construct a differentiable physical camera simulation pipeline that uses the generated initial images and extracted sensor style codes. Using the input as input, the final image is generated through a series of parameterized physical transformations. ; The simulation pipeline performs the following micronable operations in the physical sequence of real camera imaging: First, lens distortion simulation is performed, using... distortion coefficients of the encoding ,in , Represents the radial distortion coefficient. , Represents the tangential distortion coefficients for the generated image. pixel coordinates A distortion transformation is performed, and the distorted image is obtained through differentiable bilinear sampling. Subsequently, a rolling shutter simulation was performed, using... Line readout time of the Chinese encoding And camera motion parameters, for the distorted image Apply line-by-line offset to simulate the rolling shutter effect: ； in This represents the output image after simulating the rolling shutter effect. and These represent the row index and column index of the pixel, respectively, and the row offset. It is a line number A function of camera movement speed; Finally, color response and noise injection are performed using... Camera Response Function (CRF) parameters and color matrix encoded in Chinese For images Perform color transformation, and according to The noise model encoded in the code generates the corresponding noise map. Add to the image: in, This represents the final image after noise injection. This represents the image after color response transformation. This represents the generated additive noise map; S72. If the input real UAV image includes EXIF ​​metadata or precise camera calibration information Then, this information is used to determine the sensor style code. Online fine-tuning is performed to further improve the accuracy of the physical simulation; a weakly supervised calibration loss function is established: ； in, It is an encoder that maps EXIF ​​information to style code space; this loss is added to the total loss. In this process, it is optimized together with other objectives of the generator to achieve end-to-end sensor style code fine-tuning based on real camera data.

9. The method for generating UAV accident images by integrating multimodal semantic constraints and camera physical perception according to claim 8, characterized in that, Training uses joint loss: ； The weights are set as follows: , , , , , .

10. A UAV accident image generation system integrating multimodal semantic constraints and camera physical perception, characterized in that, The system for implementing the method according to any one of claims 1-9 comprises: The semantic and geometric prior construction module is used to perform panoramic segmentation and open vocabulary alignment on simulated traffic event images. It combines monocular depth and road geometric priors to generate soft semantic masks and class prototype libraries with category uncertainty weights as joint spatial and category constraints. The dual-branch generation network module is used to construct a CNN encoder-decoder for the content branch and a deformable Transformer for the global style branch. At each scale, it fuses soft mask statistics and class prototypes through semantic modulation across the attention normalization module SCAN, and then supplements it with discrete wavelet frequency domain paths to inject high-frequency details. The sensor style injection module is used to introduce the sensor style code extracted from real UAV images by the camera response encoder, including the camera response function, rolling shutter and noise spectrum. It adopts a convolution operation that integrates whitening and coloring transformation as well as weight modulation and demodulation mechanism to achieve controllable injection and stabilization of global style and sensor characteristics. The panoramic condition discrimination module is used to design a panoramic condition multi-expert discriminator, which integrates semantic embedding, geometric consistency constraints and momentum memory to achieve consistent discrimination across scales and priors; The training strategy optimization module is used to adopt a hybrid paradigm of hinged adversarial loss and dual ranking loss, and introduces an exponential moving average (EMA) teacher network and a dual timescale update strategy. The generalization enhancement module is used for semantic hierarchical masking reconstruction, semantic consistency based on text-image alignment, block-level similarity retrieval based on vision-geometric perception, and CVaR optimization to enhance out-of-domain generalization. The physical camera simulation module is used to perform physical consistency style rendering of lens distortion, rolling shutter and noise injection based on sensor style codes using a differentiable camera simulator, and output traffic accident images with realistic UAV imaging characteristics.

Images