An AI-based intelligent environment simulation method and system

Abstract

This invention discloses an intelligent environment simulation method and system based on artificial intelligence, belonging to the field of artificial intelligence environment simulation technology. The method includes: generating an initial image sequence using an optimized VAE model based on an updated initial scene and a POMDP belief state set; optimizing the initial image sequence using a U-Net diffusion model; calculating the VAE encoder reconstruction loss and the U-Net diffusion model diffusion denoising loss; calculating the joint loss function of the VAE encoder and the U-Net diffusion model; generating a high-quality image sequence using the joint loss function through Monte Carlo simulation; and optimizing the high-quality image sequence using a conditional diffusion model to generate the final scene image sequence. This achieves efficient fusion of multimodal data, overcoming the inconsistency between visual and semantic data in existing technologies. The inner loop rapidly generates task-specific parameters, while the outer loop optimizes the generalization ability of the meta-model, enabling the model to quickly adapt to edge scenes and significantly improving the generation efficiency and fidelity for rare scenes.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence environment simulation technology, and in particular to an intelligent environment simulation method and system based on artificial intelligence. Background Technology

[0002] In recent years, intelligent environment simulation technology has made significant progress in fields such as autonomous driving, virtual testing, and complex scene analysis. With the rapid development of artificial intelligence technology, deep learning-based simulation methods are gradually replacing traditional rule-based modeling methods and are widely used in scenarios such as traffic scene simulation, industrial automation testing, and virtual reality. Methods combining reinforcement learning (such as Deep Deterministic Policy Gradient, DDPG) can dynamically optimize scene parameters, enhancing the interactivity and realism of the simulation environment. Simultaneously, the introduction of meta-learning techniques enables models to quickly adapt to new tasks, improving generalization ability in diverse scenarios. However, the comprehensive application of these technologies is still in the exploratory stage, especially when dealing with complex edge scenarios (such as severe weather and sudden accidents). Existing methods have limitations in multimodal data integration, dynamic consistency, and task-specific adaptation. Although existing technologies have made progress in simulation, several shortcomings remain, particularly in generating high-quality, robust edge scene image sequences. First, existing methods often lack a unified semantic alignment mechanism when fusing multimodal data, making it difficult to balance visual fidelity and semantic consistency in the generated scenes. For example, generative models based on GANs or single VAEs often struggle with this aspect. When dealing with dynamic scenes (such as vehicle movement in rainy weather), problems such as frame discontinuity or loss of detail are prone to occur. Secondly, existing technologies are not adaptable enough to edge scenes. Traditional models have difficulty quickly adjusting parameters to generate sequences that meet specific task requirements when faced with rare accidents or extreme weather. In addition, existing methods lack systematicity in task layering and parameter optimization, making it difficult to effectively balance the multi-task requirements of object detection, dynamic modeling, and global semantic understanding. This results in poor robustness of the generated simulation environment in complex interactive scenarios. In contrast, our invention significantly improves the fidelity, dynamic consistency, and task adaptability of generated sequences in edge scenes through efficient fusion of multimodal data, meta-learning-driven adaptive parameter optimization, and a diffusion model with joint loss optimization. Summary of the Invention

[0003] In view of the aforementioned existing problems, the present invention is proposed.

[0004] Therefore, this invention provides an intelligent environment simulation method and system based on artificial intelligence, which solves the problems of lack of a unified semantic alignment mechanism when fusing multimodal data, insufficient adaptability of existing technologies to edge scenarios, and lack of systematic approach in task layering and parameter optimization.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides an intelligent environment simulation method based on artificial intelligence, which includes collecting multimodal data and preprocessing it, generating an initial scene based on the LLaMA-3-8B model, and updating the initial scene using a deep deterministic strategy gradient.

[0007] Based on the updated initial scene and POMDP belief state set, a scene tensor is formed. An initial image sequence is generated using an optimized VAE model. The initial image sequence is then optimized using a U-Net diffusion model. The reconstruction loss of the VAE encoder and the diffusion denoising loss of the U-Net diffusion model are calculated. The joint loss function of the VAE encoder and the U-Net diffusion model is also calculated. A high-quality image sequence is generated using the joint loss function through Monte Carlo simulation. Finally, the high-quality image sequence is optimized using a conditional diffusion model to generate the final scene image sequence.

[0008] Visual features are extracted from the final scene image sequence, and K-means clustering analysis is used to stratify edge cases based on visual features and semantic embedding.

[0009] An adaptive parameter is created using a meta-learning inner loop method, optimized using a meta-learning outer loop method, and then the optimized parameter is used to generate a single output sequence. The generated single output sequence is then validated using quality metrics and subjected to robustness testing.

[0010] As a preferred embodiment of the AI-based intelligent environment simulation method described in this invention, the following steps are included: collecting and preprocessing multimodal data; generating an initial scene based on the LLaMA-3-8B model; and updating the initial scene using Deep Deterministic Policy Gradient (DDPG). This involves collecting multimodal data including text, images, LiDAR, GPS, temperature, humidity, and video, preprocessing the data, generating multimodal feature vectors from the preprocessed data using a CLIP-ViT-L-336px multimodal pre-trained model, combining the feature vector list into a feature matrix A using the numpy.stack function, storing it in a Faiss vector database, training the multimodal feature vectors based on the LLaMA-3-8B pre-trained language model, outputting the initial scene in JSON format, and parsing the initial scene using a JSON parsing library. The CARLA simulator dynamically generates CARLA simulation environment vectors based on the parsed JSON scene. It uses the Transformer method to integrate the CARLA simulation environment vectors with multimodal feature vectors into a feature vector set E. The feature vector set E is used as the state input. A deep deterministic policy gradient architecture is configured using Actor and Critic neural networks. Based on the state input feature vector set E, it outputs continuous action data vectors. By utilizing replay buffer transformation and soft targets, the continuous action data vectors output by the Actor-Critic network are optimized. The fields in the JSON initial scene are modified and updated according to the optimized action data vectors.

[0011] As a preferred embodiment of the AI-based intelligent environment simulation method of the present invention, the following steps are included: calculating the VAE encoder reconstruction loss and the U-Net diffusion model diffusion denoising loss, and calculating the joint loss function; generating a high-quality image sequence using the joint loss function via Monte Carlo simulation; and generating an initial image sequence based on the latent vector z using the VAE decoder. The U-Net diffusion model is used to process the initial image sequence through a forward process. Gaussian noise is added until it becomes pure noise, and the reverse process is trained to generate an optimized initial image sequence for noise removal. Based on the initial image sequence Calculate the reconstruction loss of the VAE encoder Denoising loss of U-Net diffusion model Based on reconstruction loss and denoising loss Calculate the joint loss function of the VAE encoder and the U-Net diffusion model. Gradient descent and the AdamW optimizer are used based on the joint loss function. Calculate the weight set of VAE encoder and U-Net. Using weight sets in Monte Carlo simulation and initial image sequence and scene tensor generation of high-quality image sequences The final scene image sequence is generated using a conditional diffusion model. This yields high-quality environmental condition images.

[0012] As a preferred embodiment of the AI-based intelligent environment simulation method of the present invention, the step of extracting visual features from the final scene image sequence and using K-means clustering analysis to perform task stratification of edge cases based on visual features and semantic embedding refers to using a pre-trained ResNet-50 model from the scene image sequence. Extract high-dimensional visual feature vectors f, and use a pre-trained CLIP-ViT-L-336px model to analyze the scene template set. Encode the data and extract the semantic embedding vector from the encoding. K-means clustering analysis was used based on the visual feature vector f and the semantic embedding vector. The edge case is decomposed into three task layers. .

[0013] As a preferred embodiment of the artificial intelligence-based intelligent environment simulation method described in this invention, the step of creating adaptive parameters using the meta-learning inner loop method refers to using meta-model parameters. Initialize the transformer-UNet model by copying the meta-model parameters. For each task layer Create task-specific parameters Use the nearest neighbor retrieval method for each task layer Retrieve fixed sample support set The meta-learning inner loop method is implemented using PyTorch's AdamW optimizer, which processes scene image sequences within the inner loop. and scene tensor Inputting the Transformer model, the output sequence o of the Transformer model is compared with the scene template set. By comparing the results, calculating the task-specific loss, and performing a fixed number of gradient descent steps, this iterative process generates adaptive parameters. .

[0014] As a preferred embodiment of the AI-based intelligent environment simulation method described in this invention, the step of optimizing adaptive parameters through a meta-learning outer loop method and generating a single output sequence from the optimized parameters involves implementing the meta-learning outer loop method in PyTorch using the AdamW optimizer, comparing the output sequence with structured label data, summarizing the performance of all task layers, calculating the combined error index, and updating the meta-model parameters through the outer loop method. Meanwhile, optimize adaptive parameters The parameter fusion module integrates task-specific parameters. The encoding is converted into a unified control vector, and the control vector is linearly concatenated with the scene image sequence and the scene tensor. Combined with the transformer layer of UNet as a conditional input, it guides cross-modal fusion and spatial-semantic modeling in the model's attention mechanism, modulates the task-specific features of the generated sequence, and finally generates a single output image sequence y.

[0015] As a preferred embodiment of the artificial intelligence-based intelligent environment simulation method described in this invention, the quality index verification and robustness test of the generated single output sequence refer to comparing the single output image sequence y with structured label data using the SSIM method to obtain the SSIM structural similarity index, and setting a threshold. , as well as The pixel-level fidelity was evaluated using the scikit-image library in Python, and image sharpness was measured using PSNR (OpenCV). Furthermore, CLIP similarity (CLIP-ViT-L-336px) was used to evaluate the relationship with the scene tensor. The semantic alignment similarity score, if it simultaneously satisfies the target similarity SSIM> Generate PSNR score > Generate similarity scores > Then, after verification, using the quality-verified output sequence y, the task is evaluated using YOLOv8 with perturbations applied via the alumensations library. Target detection accuracy, generate mAP score, and set threshold. Make mAP score > The task is computed using OpenCV's Farneback method. The optical flow field is analyzed to obtain the flow variance index, and a threshold is set. Make the variance index ≤ CLIP model re-evaluates the task Semantic similarity of perturbed images, with a threshold set. To make similarity The metrics are aggregated using a weighted average to generate a robustness report confirming stability. Using the output sequence y, metrics, and robustness report, the h5py library packages the output sequence y into an HDF5 simulation package to ensure deployment compatibility with CARLA.

[0016] Secondly, the present invention provides an intelligent environment simulation system based on artificial intelligence, including a multimodal data acquisition and initial scene generation module, which provides multimodal input for scene generation, constructs an initial simulation scene, and provides a basis for subsequent dynamic optimization;

[0017] An initial image sequence generation module is used to sample the latent vectors into an initial image sequence using a VAE decoder;

[0018] A high-quality image sequence generation module is used to generate high-quality image sequences through Monte Carlo simulation, and then generate the final scene image sequence through a conditional diffusion model;

[0019] The meta-learning inner and outer loop modules are used to initialize the transformer-UNet model, generate adaptive parameters based on task-specific loss, and compare the output sequence with structured label data through the outer loop to generate the final image sequence.

[0020] The quality verification and robustness testing module ensures the high quality and robustness of the final image sequence, supporting CARLA deployment.

[0021] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the artificial intelligence-based intelligent environment simulation method described in the first aspect of the present invention.

[0022] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the artificial intelligence-based intelligent environment simulation method described in the first aspect of the present invention.

[0023] The beneficial effects of this invention are as follows: It achieves efficient fusion of multimodal data through unified semantic embedding and probability state distribution, overcoming the shortcomings of visual and semantic inconsistencies in existing technologies. The generated scene tensor not only preserves visual details but also captures the probability distribution of dynamic scenes through belief states, improving the semantic consistency of the simulation environment in complex interactive scenarios. The inner loop rapidly generates task-specific parameters, while the outer loop optimizes the generalization ability of the meta-model. This method enables the model to quickly adapt to edge scenarios, significantly improving the generation efficiency and fidelity for rare scenes compared to traditional GAN ​​models. Monte Carlo simulation combined with a conditional diffusion model generates high-quality image sequences. Through CLIP embedding and conditional guidance of belief states, it ensures the sequence adapts to the dynamic characteristics of extreme scenarios. Task hierarchical integration with meta-learning achieves multi-task collaborative optimization. Adjustment vectors enhance the model's adaptability to specific tasks, significantly improving the detection accuracy and semantic alignment of the generated sequences. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of an artificial intelligence-based intelligent environment simulation method in Example 1.

[0026] Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based intelligent environment simulation system in Example 1.

[0027] Figure 3 This is a diagram illustrating the model training process of an artificial intelligence-based intelligent environment simulation method in Example 1.

[0028] Figure 4 This is a meta-learning framework diagram of an artificial intelligence-based intelligent environment simulation method in Example 1. Detailed Implementation

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0032] Example 1, referring to Figures 1 to 4 This is the first embodiment of the present invention, which provides an intelligent environment simulation method based on artificial intelligence, including the following steps:

[0033] S1. Collect multimodal data and preprocess it, generate an initial scene based on the LLaMA-3-8B model, and update the initial scene using a deep deterministic gradient strategy.

[0034] Based on the updated initial scene and POMDP belief state set, a scene tensor is formed. An initial image sequence is generated using an optimized VAE model. The initial image sequence is then optimized using a U-Net diffusion model. The reconstruction loss of the VAE encoder and the diffusion denoising loss of the U-Net diffusion model are calculated. The joint loss function of the VAE encoder and the U-Net diffusion model is also calculated. A high-quality image sequence is generated using the joint loss function through Monte Carlo simulation. Finally, the high-quality image sequence is optimized using a conditional diffusion model to generate the final scene image sequence.

[0035] Specifically, multimodal data is collected and preprocessed. An initial scene is generated based on the LLaMA-3-8B model, and the initial scene is updated using Deep Deterministic Policy Gradient (DDPG). The collected multimodal data includes text (such as traffic rules, weather patterns, ISO26262 standards, CARLA simulation instructions), images (satellite images, traffic scenes captured by cameras), LiDAR, GPS, temperature, humidity, and video (dashcam video (720p, 30fps, capturing urban / highway scenes)). After preprocessing, the CLIP-ViT-L-336px multimodal pre-trained model is used to generate multimodal feature vectors (dimension 512) from the preprocessed data. The numpy.stack function is used to combine the list of feature vectors into a feature matrix A, which is stored in the Faiss vector database. The multimodal feature vectors are trained based on the LLaMA-3-8B pre-trained language model, and the initial scene (such as weather, traffic flow, events, spatial layout) is output in JSON format.

[0036] The initial scene is parsed using a JSON parsing library. Based on the CARLA simulator, CARLA simulation environment vectors (e.g., vehicle position, speed) are dynamically generated from the parsed JSON scene. A Transformer method is used to integrate the CARLA simulation environment vectors with multimodal feature vectors into a feature vector set E. Feature vector set E is used as the state input (the CARLA environment vectors (low-dimensional structured data) are dimensionally adjusted using a linear layer to align them with the multimodal feature vectors (512-dimensional CLIP embeddings)). A deep deterministic policy gradient architecture is configured using an Actor and Critic neural network (3-layer MLP). The state input feature vector set E (the Actor network processes the state input as a vector of real-valued numbers to produce continuous actions (e.g., acceleration, steering angle), while the Critic network processes the state and action inputs as estimates (e.g., defining a reward function as an estimate) to evaluate the quality of the actions) outputs a continuous action data vector. By utilizing replay buffer transformation and soft objectives, the continuous action data vector output by the Actor-Critic network is optimized. Based on the optimized action data vector, the fields in the JSON initial scene (specific attributes in the JSON scene, such as vehicle location, traffic flow, event outcome) are modified to update the JSON initial scene.

[0037] By generating 512-dimensional multimodal feature vectors using CLIP-ViT-L-336px and integrating them into a feature matrix, and combining this with LLaMA-3-8B to output an initial JSON scene, a unified semantic representation of text, image, sensor, and video data is achieved, overcoming the deficiency of insufficient semantic alignment. A feature vector set E is formed by integrating CARLA environment vectors and multimodal features using Transformer. Continuous actions (such as acceleration and steering angle) are generated through DDPG's Actor-Critic architecture (3-layer MLP), and the action vectors are optimized using replay buffers and soft targets. The JSON scene fields (such as vehicle position and traffic flow) are dynamically updated. This method significantly enhances the dynamic adaptability of edge scenes (such as sudden accidents and severe weather). Compared with traditional GAN ​​or VAE models, the generated initial scene is more realistic and interactive. The optimized scene provides high-quality input for subsequent image sequence generation and task hierarchicalization, effectively solving the systematic deficiencies of multi-task collaborative optimization.

[0038] Furthermore, based on the updated initial scene and the POMDP belief states set into a scene tensor, an optimized VAE model is used to generate an initial image sequence, and a U-Net diffusion model is used to optimize the initial image sequence. The reconstruction loss of the VAE encoder and the diffusion denoising loss of the U-Net diffusion model are calculated, and the joint loss function of the VAE encoder and the U-Net diffusion model is calculated. A high-quality image sequence is generated using the joint loss function through Monte Carlo simulation. The final scene image sequence is generated by optimizing the high-quality image sequence through a conditional diffusion model. The CLIP model is used to generate embedding vectors from the updated initial scene data vectors and initialize the POMDP belief states B(s). (This refers to the probability distribution of states (such as vehicle position, weather state) in the scene template, obtained through statistical inference methods. For example, if the state space contains N variables, each variable is represented by a fixed dimension (such as position 2D, speed 2D, weather 1D), then B(s) can be represented as an N-dimensional probability distribution vector). The embedding vectors and belief states are cached in a distributed database (such as Redis, stored in key-value pairs (template ID-embedded vector-belief state)). Each embedding vector and belief state is set into a scene tensor. (e.g., [1, 512+N]), each tensor contains an embedding vector of a scene template and the corresponding POMDP belief state B(s);

[0039] By integrating a temporal convolution module into an optimized VAE model, the feature matrix A is generated as a convolutional temporally enhanced feature. The VAE encoder (Encoder) adds temporal augmentation features. and scene tensor The latent vector z mapped into the VAE latent space refers to the "compressed encoding" of the input data (such as sensor sequences, video frames, scene descriptions), which retains key information (such as vehicle trajectory patterns and weather features), but has a dimension much lower than the original input, which facilitates the computation of the generation process and the modeling of dynamic characteristics. The belief state ensures that the latent vector is aligned with the probabilistic environment state.

[0040] The initial image sequence is generated based on the latent vector z using the VAE decoder. The decoder receives the latent vector z, and progressively upsamples it through transposed convolutional layers, mapping the low-dimensional vector z into a high-dimensional tensor to generate a multi-frame state sequence. Initially, each frame is a high-resolution image (e.g., RGB format) representing environmental conditions (e.g., roads, vehicles, weather).

[0041] The U-Net diffusion model is used to process the initial image sequence through a forward process. Gaussian noise is added until it becomes pure noise, and the reverse process is trained to generate an optimized initial image sequence for noise removal. :

[0042] ,

[0043] in, This represents the Gaussian noise at diffusion step t.

[0044] Calculate the reconstruction loss of the VAE encoder Denoising loss of U-Net diffusion model :

[0045] ,

[0046] ,

[0047] in, For decoder, The mathematical expectation for a given sample and time step refers to the expectation calculated in the mathematical model. and (This refers to the process of averaging the training data samples and the time steps of the diffusion model). The L2 norm of the pixel-level error. For real-state sequences, scene samples were generated using the CARLA simulation platform through data acquisition and preprocessing, and then processed by OpenCV and Librosa to extract high-resolution (512x512) dynamic sequence features.

[0048] Calculate the joint loss function of the VAE encoder and the U-Net diffusion model. :

[0049] ,

[0050] in, and The weighting coefficients are obtained through Bayesian optimization to balance VAE and diffusion loss.

[0051] Gradient descent and the AdamW optimizer are used based on a joint loss function. Calculate the weight set of VAE encoder and U-Net. This improves the quality of the generated image sequences, ensuring that the output environment is realistic and dynamically consistent.

[0052] ,

[0053] in, The learning rate is obtained from hyperparameter settings. The gradient of the loss with respect to the weights is obtained through an automatic differentiation method;

[0054] Using weight sets through Monte Carlo simulation Generate high-quality image sequences :

[0055] ,

[0056] Where P is the state transition probability, obtained by the state transition estimation method, Q is the time span (50 frames), obtained by the fixed step size allocation method, and Monte Carlo is the Monte Carlo simulation method;

[0057] The final scene image sequence is generated using a conditional diffusion model. This yields high-quality environmental state images and outputs realistic scenes.

[0058] ,

[0059] ,

[0060] in, This is a conditional diffusion guidance method. and The guiding weights were obtained through experimental optimization. CLIP embeddings for scene tensors are obtained through CLIP encoding. PyTorch is used to implement linear projection and normalization, mapping the semantic features of CLIP embeddings to the probability states of B(s) to a unified dimension. LayerNorm normalization is combined to ensure numerical compatibility. Finally, a conditional vector is generated through weighted fusion, which solves the problem of inconsistent dimensions of heterogeneous vectors and enhances semantic alignment and generation quality.

[0061] By synthesizing scene tensors from updated initial scene data and POMDP confidence states, this method uses CLIP embeddings (512-dimensional) and caches them in a Redis database to ensure robust semantic alignment across multimodal inputs, overcoming the lack of unified semantic consistency in previous methods such as GANs or independent VAEs. The optimized VAE is integrated with a temporal convolutional module, mapping feature matrices and scene tensors into latent vectors to generate initial image sequences that capture dynamic features. The diffusion model is optimized in the joint loss function by adjusting weights through AdamW to ensure high visual fidelity and temporal consistency. Monte Carlo simulations utilize state transition probabilities and optimized weights to generate high-quality sequence samples, while the conditional diffusion model generates the final sequence, tailored for extreme cases such as rare accidents or severe weather. This method significantly improves adaptability to complex scenes, unlike traditional models that struggle in dynamic edge cases. The joint optimization of VAE and U-Net losses facilitates systematic multi-task collaboration, supporting downstream tasks such as object detection and dynamic modeling, and addressing the shortcomings of previous systems in generating semantically consistent, dynamically coherent, and task-adaptive edge-case scenes.

[0062] S2. Extract visual features from the final scene image sequence, and use K-means clustering analysis to stratify the edge cases based on visual features and semantic embedding;

[0063] Use the meta-learning inner loop method to create adaptive parameters;

[0064] Specifically, visual features are extracted from the final scene image sequence, and K-means clustering analysis is used to stratify edge cases based on visual features and semantic embeddings. A pre-trained ResNet-50 model is then used to extract features from the scene image sequence. Extract high-dimensional visual feature vectors f, (using a pre-trained ResNet-50 model implemented in PyTorch to process the sequence). For each frame, a 512-dimensional feature vector is extracted to capture the visual pattern. All features from 50 frames are aggregated to form a visual feature vector f. A pre-trained CLIP-ViT-L-336px model is used to process the scene template set. Encoding is performed, the template set consists of a feature vector set E, and semantic embedding vectors (512 dimensions) are obtained from the encoding. Capture the semantic features of the template;

[0065] K-means clustering analysis (K=3, scikit-learn) was used based on the visual feature vector f and the semantic embedding vector. Edge cases (referring to challenging scenarios such as sudden obstacles, severe weather (e.g., heavy rain), and complex interactions (e.g., pedestrian crossings during collisions)) are broken down into three task layers:

[0066] ,

[0067] in, For task set For object detection tasks. For dynamics learning tasks, For global context tasks, This is a clustering algorithm. It divides the object detection task into layers based on local spatial information (such as object boundaries and texture) in the visual feature vector f, combined with semantic information related to object categories (such as "pedestrian" or "vehicle" labels) from semantic embedding. The corresponding task is to evaluate the object detection accuracy using the YOLOv8 model. This involves dividing the temporal information (such as inter-frame displacement and optical flow features) in the visual feature vector f into dynamic learning task layers, combined with semantic information related to dynamic events in the semantic embedding (such as "vehicle acceleration" or "pedestrian crossing"). The corresponding task is to use OpenCV's Farneback method to calculate the optical flow field, obtain the flow variance index, and divide the global semantic information (such as "heavy rain weather" or "high-traffic intersection") in the semantic embedding vector into a global context task layer, combined with the scene background information (such as road structure and sky color) in the visual feature vector f. The corresponding task is to evaluate the semantic similarity of perturbed images using the CLIP-ViT-L-336px model.

[0068] By extracting visual feature vectors and generating semantic embeddings from scene template sets using CLIP-ViT-L-336px, this method captures detailed visual patterns and contextual semantic information. K-means clustering analysis decomposes edge cases into three task layers: object detection, dynamic learning, and global context, corresponding to YOLOv8-based detection, Farneback optical flow analysis, and CLIP-based semantic similarity evaluation, respectively. This task layering overcomes the semantic misalignment of previous methods (such as GAN and VAE) by integrating visual and semantic features. Targeted task optimization enhances adaptability to challenging scenes, and adjusting detection accuracy (mAP), dynamic consistency (flow variance), and semantic robustness (CLIP similarity) ensures multi-task collaboration of the system, significantly improving simulation fidelity and robustness. It can accurately handle edge cases and support downstream tasks such as object detection and trajectory prediction.

[0069] Furthermore, creating adaptive parameters using the meta-learning inner loop method refers to using meta-model parameters. (Load the denoising U-Net model from Stable Diffusion (Rombach et al., 2022) into PyTorch, using pre-trained weights as meta-model parameters) The weights are trained on a large-scale image dataset (suitable for high-quality vision generation), and the transformer-UNet model is initialized by replicating the meta-model parameters. For each task layer Create task-specific parameters ,(parameter By copying metamodel parameters Obtain, such as Related , Related , Related (linked together by index number)

[0070] Use the nearest neighbor search method for each task layer Retrieve fixed sample support set The meta-learning inner loop method is implemented using PyTorch's AdamW optimizer, which processes scene image sequences within the inner loop. and scene tensor Inputting the Transformer model, the output sequence o of the Transformer model is compared with the scene template set. Compare the results, calculate task-specific losses, quickly adapt to the task, and update the model parameters. Within the inner loop, a fixed number of gradient descent steps are performed (e.g., 5 iterations), and this iterative process generates adaptive parameters. (Adaptive parameters) By copying model parameters After 5 iterations, the gradient is calculated based on the loss between the predicted and true sequences, and the parameters are updated using the AdamW optimizer. AdamW utilizes adaptive learning rate and weight decay to ensure efficient updates, ultimately generating task-specific parameters that can quickly adapt to edge scenarios (such as rare accidents or severe weather).

[0071] By initializing the Transformer-UNet model with pre-trained, stable, diffused U-Net weights and replicating the meta-model parameters for task-specific parameters for each task layer, this approach ensures robust initialization across different edge scenarios. The meta-learning inner loop uses nearest neighbor retrieval to obtain a fixed support set for each task layer. Five gradient descent iterations are performed using PyTorch's AdamW optimizer to compute the task-specific loss between the Transformer output sequence and the scene template set. This process iteratively updates parameters to generate adaptive parameters tailored to edge cases such as rare accidents or severe weather, overcoming the adaptability limitations of traditional GAN ​​or VAE models. The AdamW optimizer's adaptive learning rate and weight decay ensure efficient convergence and enhance task-specific performance. This systematic approach supports multi-task optimization by generating parameters that meet specific simulation requirements.

[0072] S3. Optimize the adaptive parameters through the meta-learning outer loop method, and generate a single output sequence from the optimized parameters. Then, verify the quality indicators and test the robustness of the generated single output sequence.

[0073] Specifically, the meta-learning outer loop method optimizes adaptive parameters, and the optimized parameters are used to generate a single output sequence. The AdamW optimizer is used to implement the meta-learning outer loop method in PyTorch. The output sequence is compared with structured label data (obtained through CARLA simulation and multimodal perception processing (OpenCV)) to provide optimization direction for the outer loop. This guides the meta-model to improve its generalization ability to new edge scenarios through outer loop optimization. The performance of all task layers is summarized (referring to the performance of each task layer on its task, such as mAP (detection), optical flow variance (dynamic), and CLIP similarity (global semantics)). A combined error metric is calculated (the total error metric obtained by standardizing and weighting multiple performance metrics, used to evaluate the overall performance of the entire model in multi-task scenarios). The meta-model parameters are then updated using the outer loop method. (Summarize the combined error metric (mAP), optical flow variance, and CLIP similarity weighted average of each task layer), calculate the global loss, use the AdamW optimizer to perform gradient descent based on the global loss, adjust the Transformer weights, reduce aggregation error, and improve generalization ability), and simultaneously optimize adaptive parameters. Maintain task-specific performance by integrating insights from meta-model updates; integrate task-specific parameters via a parameter fusion module. Encoded as a uniform control vector (including all task-specific parameters). ,and and The control vector is then linearly concatenated with the scene image sequence and the scene tensor. Combined with the transformer layer of UNet as a conditional input, it guides cross-modal fusion and spatial-semantic modeling in the model's attention mechanism, modulates the task-specific features of the generated sequence (such as enhancing pedestrian detection accuracy or vehicle dynamic consistency in rainy weather), and finally generates a single output image sequence y, which is adapted to extreme cases of rare accidents and severe weather, and achieves adaptive generation with high consistency and high fidelity.

[0074] By implementing a meta-learning outer loop using PyTorch's AdamW optimizer, this method compares the output sequences based on structured label data from CARLA simulation and OpenCV processing, calculating a composite error metric to guide global loss calculation. This enables Transformer weights, enhancing the meta-model's generalization to new edge scenarios and surpassing the adaptability limitations of traditional GAN ​​or VAE models. Simultaneously, task-specific parameters are optimized to maintain performance in object detection, dynamic learning, and global context tasks. The parameter fusion module encodes these parameters into a unified control vector, linearly connected to the scene image and tensors, and injects it into TransUNet's Transformer layers to guide cross-modal fusion and spatial semantic modeling. This generates a single output sequence. Through enhanced task-specific functionality, this method effectively alleviates the shortcomings of existing intelligent environment simulation techniques, specifically addressing issues such as inconsistent semantic alignment, poor adaptability to edge situations, and insufficient multi-task optimization.

[0075] Furthermore, the generated individual output sequences are validated using quality metrics and tested for robustness. Specifically, the SSIM method is used to compare the individual output image sequence y with the structured label data to obtain the SSIM structural similarity index, and a threshold is set. , as well as (Pixel-level fidelity was evaluated using perceptual fidelity thresholding and semantic perceptual thresholding, respectively.) The scikit-image library in Python was used to evaluate the image fidelity, and PSNR (OpenCV) was used to measure image sharpness. CLIP similarity (CLIP-ViT-L-336px) was used to evaluate the image's relationship with the scene tensor. The semantic alignment similarity score, if it simultaneously satisfies the target similarity SSIM> Generate PSNR score > Generate similarity scores > The verification confirmed the quality of the output image sequence y in terms of visual fidelity (pixel-level consistency) and semantic fidelity (scene semantic consistency).

[0076] Using the quality-verified output sequence y, perturbations (e.g., Gaussian noise, brightness variations) are applied using the alumniation library, and the evaluation task is performed using YOLOv8. The target detection accuracy is used to generate an mAP score, and the threshold is set by the detection performance threshold optimization method. Make mAP score > (For mAP, the YOLOv8 model is fine-tuned on the CARLA edge case dataset, targeting objects such as pedestrians and vehicles, with weighted IoU to enhance the detection of complex objects, data augmentation through whitening (e.g., Gaussian noise, brightness variations) to improve robustness, and non-maximum suppression (NMS) to filter low-quality bounding boxes to ensure mAP scores > 0.5%.) The task is computed using OpenCV's Farneback method. The optical flow field is analyzed to obtain the flow variance index, and a threshold is set using the motion consistency assessment method. Make the variance index ≤ OpenCV's Farneback optical flow method achieves a smooth flow field by adjusting the window size and pyramid level optimization. Flow consistency loss is integrated into TransUNet training to limit frame-to-frame motion, while Gaussian filtering reduces local noise and ensures that the variance index is ≤ CLIP model re-evaluates the task. The semantic similarity of the perturbed images (by setting a threshold through Perturbation Semantic Stability Analysis). Through semantic stability analysis under perturbation, similarity is improved. The CLIP-ViT-L-336px model evaluates perturbed images and combines fine-tuning and semantic stability loss to maintain alignment with the scene tensor, achieving similarity. The metrics (mAP score, flow variance metric, semantic similarity) are aggregated by weighted average to generate a robustness report that confirms stability. Using the output sequence y, metrics, and robustness report, the h5py library packages the output sequence y into an HDF5 simulation package to ensure deployment compatibility with CARLA. This step ultimately determines the intelligent environment simulation by ensuring that the adjusted sequence is both high-quality and robust.

[0077] By validating individual output sequences based on structured label data using SSIM, PSNR, and CLIP similarity, this method ensures high visual fidelity (pixel-level consistency) and semantic alignment, overcoming the semantic misalignment problem commonly found in previous GAN-based methods. Robustness testing involves applying perturbations through whitening, evaluating mAP using a fine-tuned YOLOv8 model, assessing flow variance using OpenCV's Farneback method, and evaluating semantic similarity using CLIP to ensure that all metrics meet standards. These metrics are aggregated through weighted averages to generate a comprehensive robustness report. This rigorous validation and robustness framework enhances the reliability of simulations in edge cases, surpassing the adaptability limitations of traditional methods, and supports simulation environment testing by ensuring consistent, high-quality output across different scenarios.

[0078] This embodiment also provides an intelligent environment simulation system based on artificial intelligence, including:

[0079] The multimodal data acquisition and initial scene generation module is used to provide multimodal input for scene generation, construct a preliminary simulation scene, and provide a foundation for subsequent dynamic optimization;

[0080] An initial image sequence generation module is used to sample the latent vectors into an initial image sequence using a VAE decoder;

[0081] A high-quality image sequence generation module is used to generate high-quality image sequences through Monte Carlo simulation, and then generate the final scene image sequence through a conditional diffusion model;

[0082] The meta-learning inner and outer loop modules are used to initialize the transformer-UNet model, generate adaptive parameters based on task-specific loss, and compare the output sequence with structured label data through the outer loop to generate the final image sequence.

[0083] The quality verification and robustness testing module ensures the high quality and robustness of the final image sequence, supporting CARLA deployment.

[0084] This embodiment also provides a computer device applicable to an artificial intelligence-based intelligent environment simulation method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the artificial intelligence-based intelligent environment simulation method proposed in the above embodiment.

[0085] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0086] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the artificial intelligence-based intelligent environment simulation method and system proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0087] In summary, this invention achieves efficient fusion of multimodal data through unified semantic embedding and probability state distribution, overcoming the shortcomings of visual and semantic inconsistencies in existing technologies. The generated scene tensor not only preserves visual details but also captures the probability distribution of dynamic scenes through belief states, improving the semantic consistency of the simulation environment in complex interactive scenarios. By rapidly generating task-specific parameters through an inner loop and optimizing the generalization ability of the meta-model through an outer loop, this method enables the model to quickly adapt to edge scenarios. Compared with traditional GAN ​​models, it significantly improves the generation efficiency and fidelity for rare scenarios. Monte Carlo simulation combined with a conditional diffusion model generates high-quality image sequences. Through CLIP embedding and conditional guidance of belief states, it ensures that the sequences adapt to the dynamic characteristics of extreme scenarios. The combination of task layering and meta-learning achieves multi-task collaborative optimization. The control vector enhances the model's adaptability to specific tasks, significantly improving the detection accuracy and semantic alignment of the generated sequences.

[0088] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for simulating intelligent environments based on artificial intelligence, characterized in that: include, Multimodal data is collected and preprocessed. An initial scene is generated based on the LLaMA-3-8B model and updated using a deep deterministic gradient strategy. The initial scene refers to structured text data, which uses key-value pairs to describe environmental information. Based on the updated initial scene and the POMDP belief state set, a scene tensor is formed. An optimized VAE model is used to generate an initial image sequence, and a U-Net diffusion model is used to optimize the initial image sequence. The reconstruction loss of the VAE encoder and the diffusion denoising loss of the U-Net diffusion model are calculated, and the joint loss function is calculated. A high-quality image sequence is generated using the joint loss function through Monte Carlo simulation. The final scene image sequence is generated by optimizing the high-quality image sequence through a conditional diffusion model. The POMDP belief state refers to an N-dimensional probability distribution vector, and the set refers to the concatenation of the updated initial scene and the POMDP belief state. Visual features are extracted from the final scene image sequence, and K-means clustering analysis is used to stratify edge cases based on visual features and semantic embedding. An adaptive parameter is created using a meta-learning inner loop method, optimized using a meta-learning outer loop method, and then the optimized parameter is used to generate a single output sequence. The generated single output sequence is then validated using quality metrics and subjected to robustness testing.

2. The intelligent environment simulation method based on artificial intelligence as described in claim 1, characterized in that: The process involves collecting and preprocessing multimodal data, generating an initial scene based on the LLaMA-3-8B model, and updating the initial scene using Deep Deterministic Policy Gradient (DDPG). This process includes collecting multimodal data such as text, images, LiDAR, GPS, temperature, humidity, and video, preprocessing it, using a CLIP-ViT-L-336px multimodal pre-trained model to generate multimodal feature vectors from the preprocessed data, combining the feature vector list into a feature matrix A using the numpy.stack function, and storing it in the Faiss vector database. The process also involves training multimodal feature vectors based on an LLaMA-3-8B pre-trained language model, outputting the initial scene in JSON format, and parsing the initial scene using a JSON parsing library. The CARLA simulator dynamically generates CARLA simulation environment vectors based on the parsed JSON scene. It uses the Transformer method to integrate the CARLA simulation environment vectors with multimodal feature vectors into a feature vector set E. The feature vector set E is used as the state input. A deep deterministic policy gradient architecture is configured using Actor and Critic neural networks. Based on the state input feature vector set E, it outputs continuous action data vectors. By utilizing replay buffer transformation and soft targets, the continuous action data vectors output by the Actor-Critic network are optimized. The fields in the JSON initial scene are modified and updated according to the optimized action data vectors.

3. The intelligent environment simulation method based on artificial intelligence as described in claim 2, characterized in that: The calculation of VAE encoder reconstruction loss and U-Net diffusion model diffusion denoising loss, along with the calculation of joint loss function, involves generating high-quality image sequences using Monte Carlo simulation with the joint loss function. This includes optimizing the VAE model by integrating a temporal convolution module to generate convolutional temporal enhancement features from feature matrix A. The VAE encoder adds temporal augmentation features. The scene tensor is mapped to a latent vector z in the VAE latent space, and the VAE decoder generates an initial image sequence based on the latent vector z. The U-Net diffusion model is used to process the initial image sequence through a forward process. Gaussian noise is added until it becomes pure noise, and the reverse process is trained to generate an optimized initial image sequence for noise removal. Based on the initial image sequence Calculate the reconstruction loss of the VAE encoder Denoising loss of U-Net diffusion model Based on reconstruction loss and denoising loss Calculate the joint loss function of the VAE encoder and the U-Net diffusion model. Gradient descent and the AdamW optimizer are used based on the joint loss function. Calculate the weight set of VAE encoder and U-Net. Using weight sets in Monte Carlo simulation and initial image sequence and scene tensor generation of high-quality image sequences The final scene image sequence is generated using a conditional diffusion model. This yields high-quality environmental condition images.

4. The intelligent environment simulation method based on artificial intelligence as described in claim 3, characterized in that: The process of extracting visual features from the final scene image sequence and using K-means clustering analysis to stratify edge cases based on visual features and semantic embeddings refers to using a pre-trained ResNet-50 model from the scene image sequence. Extract high-dimensional visual feature vectors f, and use a pre-trained CLIP-ViT-L-336px model to analyze the scene template set. Encode the data and extract the semantic embedding vector from the encoding. K-means clustering analysis was used based on the visual feature vector f and the semantic embedding vector. The edge case is decomposed into three task layers. .

5. The intelligent environment simulation method based on artificial intelligence as described in claim 4, characterized in that: The method of creating adaptive parameters using meta-learning inner loop refers to using meta-model parameters. Initialize the transformer-UNet model by copying the meta-model parameters. For each task layer Create task-specific parameters Use the nearest neighbor retrieval method for each task layer Retrieve fixed sample support set The meta-learning inner loop method is implemented using PyTorch's AdamW optimizer, which processes scene image sequences within the inner loop. and scene tensor Inputting the Transformer model, the output sequence o of the Transformer model is compared with the scene template set. By comparing the results, calculating the task-specific loss, and performing a fixed number of gradient descent steps, this iterative process generates adaptive parameters. .

6. The artificial intelligence-based intelligent environment simulation method as described in claim 5, characterized in that: The optimization of adaptive parameters through a meta-learning outer loop method, and the generation of a single output sequence from the optimized parameters, refers to implementing the meta-learning outer loop method in PyTorch using the AdamW optimizer. The output sequence is compared with structured label data, the performance of all task layers is summarized, the combined error index is calculated, and the meta-model parameters are updated using the outer loop method. Meanwhile, optimize adaptive parameters The parameter fusion module integrates task-specific parameters. The encoding is converted into a unified control vector, and the control vector is linearly concatenated with the scene image sequence and the scene tensor. Combined with the transformer layer of UNet as a conditional input, it guides cross-modal fusion and spatial-semantic modeling in the model's attention mechanism, modulates the task-specific features of the generated sequence, and finally generates a single output image sequence y.

7. The artificial intelligence-based intelligent environment simulation method as described in claim 6, characterized in that: Quality index verification and robustness testing of the generated individual output sequences involve comparing the individual output image sequence y with structured label data using the SSIM method to obtain the SSIM structural similarity index, and setting a threshold. , as well as The pixel-level fidelity was evaluated using the scikit-image library in Python, and image sharpness was measured using PSNR. CLIP similarity was also used to evaluate the scene tensor. The semantic alignment similarity score, if it simultaneously satisfies the target similarity SSIM> Generate PSNR score > Generate similarity scores > Then, after verification, using the quality-verified output sequence y, the task is evaluated using YOLOv8 with perturbations applied via the alumensations library. Target detection accuracy, generate mAP score, and set threshold. Make mAP score > The task is computed using OpenCV's Farneback method. The optical flow field is analyzed to obtain the flow variance index, and a threshold is set. Make the variance index ≤ CLIP model re-evaluates the task Semantic similarity of perturbed images, with a threshold set. To make similarity The metrics are aggregated using a weighted average to generate a robustness report confirming stability. Using the output sequence y, metrics, and robustness report, the h5py library packages the output sequence y into an HDF5 simulation package to ensure deployment compatibility with CARLA.

8. An intelligent environment simulation system based on artificial intelligence, based on the intelligent environment simulation method based on artificial intelligence as described in any one of claims 5 to 7, characterized in that: include, The multimodal data acquisition and initial scene generation module is used to provide multimodal input for scene generation, construct a preliminary simulation scene, and provide a foundation for subsequent dynamic optimization; An initial image sequence generation module is used to sample the latent vectors into an initial image sequence using a VAE decoder; A high-quality image sequence generation module is used to generate high-quality image sequences through Monte Carlo simulation, and then generate the final scene image sequence through a conditional diffusion model; The meta-learning inner and outer loop modules are used to initialize the transformer-UNet model, generate adaptive parameters based on task-specific loss, and compare the output sequence with structured label data through the outer loop to generate the final image sequence. The quality verification and robustness testing module ensures the high quality and robustness of the final image sequence, supporting CARLA deployment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the artificial intelligence-based intelligent environment simulation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the artificial intelligence-based intelligent environment simulation method according to any one of claims 1 to 7.

Images