Image generation method and electronic device

Abstract

The application discloses an image generation method and an electronic device. The application relates to the technical field of automatic driving and artificial intelligence, and the method comprises the following steps: determining a plurality of semantic elements in a driving scene based on a target description text in an image generation request; acquiring a plurality of first images from a preset database, wherein the plurality of first images are matched with one of the plurality of semantic elements; fusing first image features extracted from the plurality of first images and text features extracted from the target description text by using a multi-modal alignment module to generate a fusion result; and generating a target image corresponding to the image generation request based on the fusion result and the target description text. The application solves the technical problem of low image generation accuracy in the related art.

Description

Technical Field

[0001] This application relates to the fields of autonomous driving technology and artificial intelligence technology, and more specifically, to an image generation method and electronic device. Background Technology

[0002] To alleviate the scarcity of data in long-tail scenarios, current autonomous driving technologies often employ manual annotation of sparse samples or use large models based on generic text prompts to generate images. However, these methods struggle to accurately model multiple elements in complex scenes, leading to issues such as distorted target shapes, incorrect attribute binding, and inconsistent background logic, resulting in low image generation accuracy in these technologies.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides an image generation method and electronic device to at least solve the technical problem of low image generation accuracy in related technologies.

[0005] According to one aspect of the embodiments of this application, an image generation method is provided, comprising: determining multiple semantic elements in a driving scene based on target description text in an image generation request; acquiring multiple first images from a preset database, wherein the multiple first images are respectively matched with one of the multiple semantic elements; fusing the first image features extracted from the multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate a fusion result; and generating a target image corresponding to the image generation request based on the fusion result and the target description text.

[0006] Furthermore, the multimodal alignment module is used to fuse the first image features extracted from multiple first images and the text features extracted from the target description text to generate a fusion result, including: performing positional encoding on the text features to obtain encoded features; performing self-attention processing on the encoded features to obtain attention features; and performing cross-attention processing on the first image features and attention features to generate a fusion result.

[0007] Furthermore, the multimodal alignment module is obtained by training the initial alignment module using the first training data, which includes: sample images, multiple sample elements, and sample features of the sample images, wherein the multiple sample elements are obtained by semantic parsing of the sample images; or, the multimodal alignment module is obtained by jointly training the initial alignment module and the image generation module using the second training data, which includes: training samples, sample description text corresponding to the training samples, and the image generation module is used to generate a target image based on the fusion result and the target description text.

[0008] Further, obtaining multiple first images from a preset database includes: matching each semantic element among multiple semantic elements with the semantic tags of each image in the preset database to obtain a set of images corresponding to the semantic element, wherein the semantic tags of the images are obtained by semantic parsing the images; and obtaining multiple first images based on the set of images corresponding to multiple semantic elements.

[0009] Furthermore, based on the fusion result and the target description text, the target image corresponding to the image generation request is generated, including: using the fusion result as a control condition to guide the image generation model to generate the target image based on the target description text.

[0010] Furthermore, the method also includes: determining a reference image based on an image generation request; obtaining a second image that matches the reference image from a preset database; and using a multimodal alignment module to fuse the first image features extracted from multiple first images and the text features extracted from the target description text to generate a fusion result, including: using a multimodal alignment module to fuse the first image features, the second image features extracted from the second image, and the text features to generate a fusion result.

[0011] Furthermore, the method also includes: acquiring a set of driving data uploaded by multiple vehicles, wherein the driving data set is uploaded by multiple vehicles when abnormal events in the driving environment are detected during driving; acquiring multiple preset images from the driving data set, wherein the perceptual confidence of the multiple preset images is less than a preset confidence level, and / or the recognition accuracy is less than a preset accuracy level; performing semantic parsing on the multiple preset images based on at least one semantic dimension to generate semantic labels for the multiple preset images, wherein the at least one semantic dimension includes at least one of the following: environmental variables of the driving environment, type, attributes, and behavior of targets in the driving environment; and updating the preset database based on the multiple preset images and the semantic labels of the multiple preset images.

[0012] Furthermore, multiple preset images are acquired from the driving data set, including: acquiring multiple preset images from multiple original images based on one or a combination of perceptual confidence, consistency verification results, event correlation, anomaly detection results, and human feedback results from multiple original images in the driving data set; wherein, perceptual confidence is used to characterize the confidence of multiple vehicles in perceiving each original image; consistency verification results are obtained by verifying the temporal consistency of each original image using a first verification model, and by comparing the output of the first verification model with the output of at least one second verification model, the output of the first verification model being obtained by verifying the original image using the first verification model, and the output of at least one second verification model being obtained by verifying the original image using at least one second verification model; event correlation is obtained based on the time deviation between the acquisition time of each original image and the occurrence time of the abnormal event; anomaly detection results are obtained by detecting anomalies based on the image features of each original image; and human feedback results are obtained by the target object verifying the original image after each original image is output to the target object.

[0013] Furthermore, the method also includes: generating evaluation prompts for the target image based on at least one evaluation metric corresponding to the target image, wherein different evaluation metrics are used to characterize the conditions for evaluating the generation quality of the target image from different dimensions; inputting the target image and evaluation prompts into an image evaluation model, using the image evaluation model to evaluate the target image, and obtaining a target evaluation result for the target image, wherein the target evaluation result is used to characterize whether the target image meets at least one evaluation metric; determining the target type of the target image based on the target evaluation result; and adjusting the image generation model based on the target adjustment strategy corresponding to the target type; wherein, in the case of the target type being the first type, the target adjustment strategy is to train the image generation model using the target image; in the case of the target type being the second type, the target adjustment strategy is to analyze the defects of the image generation model using the target image, obtain the analysis results, and adjust the image generation model based on the analysis results; in the case of the target type being the third type, the target adjustment strategy is to manually review the target image, obtain the manual review results, and adjust the image generation model using the target image based on the manual review results.

[0014] Furthermore, the method also includes: when the target type is type one, performing multi-level annotation on the target image to obtain target annotation results for the driving scene, wherein different levels of annotation processes result in different levels of accuracy in annotating the target image; adjusting the model parameters of the vehicle perception model based on the target image and the target annotation results to obtain an adjusted perception model; testing the adjusted perception model to obtain test results for the adjusted perception model; and deploying the adjusted perception model on at least one vehicle based on the test results; wherein performing multi-level annotation on the target image to obtain target annotation results for the driving scene includes: using a target detection model to annotate the target image... Note: Initial annotation results are obtained; based on the initial annotation results, the target segmentation model is guided to annotate the target image to obtain target annotation results, wherein the accuracy of the target annotation results is greater than the accuracy of the initial annotation results; based on the test results, the adjusted perception model is deployed on at least one vehicle, including: deploying the adjusted perception model on multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or deploying the adjusted perception model on some of the multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or running the adjusted perception model on multiple vehicles, and the output of the adjusted perception model is not coupled with the control link of the multiple vehicles.

[0015] According to another aspect of the embodiments of this application, an image generation apparatus is also provided, comprising: a determining module, configured to determine multiple semantic elements in a driving scene based on target description text in an image generation request; an acquiring module, configured to acquire multiple first images from a preset database, wherein the multiple first images are respectively matched with one of the multiple semantic elements; a fusing module, configured to fuse the first image features extracted from the multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate a fusing result; and a generating module, configured to generate a target image corresponding to the image generation request based on the fusing result and the target description text.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0017] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0020] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0021] This application proposes an image generation method. First, based on the target description text in the image generation request, multiple semantic elements in a driving scene are determined. Next, multiple first images matching one of the semantic elements are retrieved from a preset database. Then, a multimodal alignment module is used to fuse the first image features extracted from the multiple first images with the text features extracted from the target description text, generating a fusion result. Finally, based on the fusion result and the target description text, the target image corresponding to the image generation request is generated. This application decomposes the driving scene into multiple independently matchable semantic elements based on the target description text, and retrieves the first images corresponding to each semantic element from the preset database as the generation basis. By using the first image features of multiple first images as structural priors and fusing them with text features, the generation process avoids the semantic vagueness and structural conjecture caused by relying solely on abstract text descriptions. Instead, it relies on visual patterns in real data for constrained reconstruction, achieving the technical objective of improving the consistency between the generated image and the real driving scene in terms of target shape, spatial relationship, and scene consistency. This achieves a dual enhancement in semantic accuracy and visual realism of the generated image, thus solving the technical problem of low image generation accuracy in related technologies. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a flowchart of an image generation method according to an embodiment of this application;

[0024] Figure 2 This is an execution flowchart of an image generation system according to an embodiment of this application;

[0025] Figure 3 This is a flowchart of a fusion processing method according to an embodiment of this application;

[0026] Figure 4 This is an optional image generation flowchart according to an embodiment of this application;

[0027] Figure 5 This is a flowchart illustrating an image generation process in a mining area scenario according to an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of an image generation apparatus according to an embodiment of this application; Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] According to an embodiment of this application, an embodiment of an image generation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] Figure 1 This is a flowchart of an image generation method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0033] Step S102: Based on the target description text in the image generation request, determine multiple semantic elements in the driving scene.

[0034] The aforementioned image generation request refers to a structured instruction issued by a user to request image generation. The types of image generation requests can include, but are not limited to, basic requests, composite requests, and structured requests. Basic requests have a single semantic meaning, containing only the target and environment; composite requests can include multiple targets, attributes, behaviors, environmental variables, etc.; structured requests can use a predefined structure to facilitate request parsing and retrieval matching. The specific image generation request needs to be determined based on user requirements. Image generation requests can be used as input trigger signals to initiate the image generation process and activate the data augmentation process. Image generation requests can also be used to clarify the semantic boundaries of the generated target to avoid the generated results being out of touch with real-world long-tail scenarios.

[0035] The aforementioned target description text can refer to a semantic subset of the target object (i.e., the entity to be detected, identified, or avoided) in the image generation request. Target description text can include, but is not limited to, target type, target attributes, target behavior, and spatial relationships; the specific target description text needs to be determined based on the actual image generation request. Target description text can be used to characterize the visual attributes and dynamic behavior of the target, providing the core basis for "what to generate," and supporting high-precision automatic annotation and quality assessment.

[0036] The aforementioned driving scenario refers to the complete visual and semantic context encountered in a real road environment. A driving scenario is defined by a set of environmental variables, which may include, but are not limited to, time, weather, lighting, geographic type, and road features. The environmental variables mentioned above are merely examples; specific environmental variables need to be determined based on actual needs. Driving scenarios can provide macroscopic semantic constraints for image generation, preventing the generation of unreasonable scenarios. They can also serve as anchor points for basic image library retrieval, matching the closest real-world environment samples; and they can be used as a basis for evaluating the quality of generated images.

[0037] The aforementioned semantic elements refer to the smallest semantic units that constitute the target description text and the driving scenario. Semantic elements may include, but are not limited to, environmental semantic elements, target semantic elements, and interaction semantic elements. Environmental semantic elements may include, but are not limited to, elements such as time, weather, lighting, and geography; target semantic elements may include, but are not limited to, elements such as target type, target attributes, and target behavior; and interaction semantic elements may include, but are not limited to, elements such as spatial relationships and interference relationships. Specific semantic elements need to be determined based on the target description text and the driving scenario. Semantic elements can be used to decompose complex descriptions into independent and manageable units, thereby improving the controllability of generated images; they can also be encoded into feature vectors to support vector indexing and retrieval, thus achieving accurate extraction through multi-image fusion.

[0038] In one optional embodiment, based on the target description text in the image generation request, the target type, attributes, behaviors, and spatial relationships described in the target description text are extracted one by one through preset semantic parsing rules. These are then matched item by item with a predefined semantic element library, and expanded and calibrated by incorporating implicit environmental conditions (such as time, lighting, and weather) to form a set of structured, atomized semantic elements of the driving scene, i.e., multiple semantic elements. This process, through fine-grained semantic decomposition and rule-based mapping, achieves accurate transformation from natural language description to computable semantic units, effectively improving the completeness and consistency of semantic expression. It provides a highly reliable and reusable input foundation for subsequent multi-image retrieval and feature fusion based on semantic elements, ensuring precise alignment of the generated scene in terms of target composition and environmental logic.

[0039] In one optional embodiment, based on the target description text in the image generation request, the text content is input into a pre-trained semantic understanding model to automatically identify and extract entities and their attributes related to the driving scenario, including target category, state features, action behavior, and spatial relationships. Implicit environmental parameters are inferred from the context, and after semantic normalization, these are mapped to a standardized semantic element space, forming a set of structured, indexable, multi-dimensional semantic elements, i.e., multiple semantic elements. This approach achieves robust parsing of complex, unstructured descriptions through end-to-end semantic embedding and context-related modeling, improving the generalization ability and coverage of semantic element extraction. This provides more accurate and scene-adaptive semantic representations for subsequent semantic matching-based image generation, enhancing the semantic consistency between the generated results and real-world long-tail scenes.

[0040] Step S104: Obtain multiple first images from a preset database, wherein each of the multiple first images is matched with one of the multiple semantic elements.

[0041] The aforementioned pre-built database refers to a collection of high-quality images and semantic tag pairs that are pre-constructed and structurally stored for specific driving scenarios. Each image in the pre-built database has corresponding semantic elements; therefore, the corresponding image can be determined from the pre-built database based on the semantic elements. The pre-built database can be used to provide searchable and reusable visual-semantic anchors for image generation, supporting accurate mapping from semantic elements to visual features.

[0042] The aforementioned first image can refer to an image retrieved from a pre-defined database based on semantic element matching principles. Each first image corresponds to one and only one visual sample of an atomic semantic element. The first image can be used to provide independent, high-fidelity visual primitives for subsequent multi-image feature fusion, avoiding feature confusion caused by a single image carrying too much semantic meaning.

[0043] In one optional embodiment, based on multiple semantic elements, multiple first images corresponding one-to-one with the multiple semantic elements are retrieved from a preset database. Each image is pre-annotated with a visual representation containing only one atomic semantic element. Through the matching relationship between semantic elements and image labels, a single-element mapping from abstract semantics to specific visual samples is achieved. This process ensures that each first image carries only a single semantic feature, avoiding visual interference caused by multiple semantic mixtures. It provides pure, independent, and high-fidelity visual primitives for subsequent multi-image feature fusion, thereby improving the alignment accuracy between semantic elements and visual content and enhancing the controllable decomposition and precise recombination capability of complex scene elements during the generation process.

[0044] In one optional embodiment, similarity retrieval is performed from a pre-defined database based on vector embeddings of semantic elements. The single image closest to each semantic element in the feature space is obtained as the first image. Each first image establishes a one-to-one semantic-visual association with the vector representation of the corresponding semantic element through a multimodal encoder, ensuring that the image content accurately reflects a single semantic dimension in terms of visual features. This approach establishes an implicit mapping between semantics and images through end-to-end learning, improving the database's generalization matching ability on novel or unforeseen semantic elements, while ensuring the visual purity and semantic specificity of the retrieval results, providing a reliable and adaptive visual input foundation for subsequent multi-image fusion.

[0045] Step S106: The first image features extracted from multiple first images and the text features extracted from the target description text are fused using the multimodal alignment module to generate a fusion result.

[0046] The aforementioned multimodal alignment module can refer to a neural network structure used to achieve multimodal feature alignment. Types of multimodal alignment modules may include, but are not limited to, neural networks based on cross-attention mechanisms, cross-modal attention layers based on transformers, dual-stream encoder-alignment heads, and feature interaction modules driven by graph neural networks. The multimodal alignment modules described above are merely examples; the specific multimodal alignment module needs to be determined based on actual design requirements. Multimodal alignment modules can be used to align and interact features from different modalities (images and text) in the semantic space, achieving dynamic association and complementary enhancement between visual features and linguistic semantics. Multimodal alignment modules can serve as front-end feature fusion units and can be trained independently of the back-end generative model.

[0047] The aforementioned text features can refer to the high-dimensional vector representation obtained by encoding the target descriptive text. Text features can be used for alignment and fusion with image features. Text features can be extracted from the target descriptive text by text encoders such as pre-trained Bidirectional Encoder Representations from Transformers (BERT) and Contrastive Language–Image Pretraining (CLIP).

[0048] The aforementioned fusion result refers to the result obtained after the multimodal alignment module fuses the first image features and text features. The fusion result can be used to selectively fuse multiple dispersed image features according to textual semantics, forming a comprehensive feature vector with semantic consistency and visual richness. The fusion result may include, but is not limited to, weighted fusion, attention masking, and hierarchical aggregation fusion results; the specific fusion result needs to be determined based on the actual fusion strategy. The fusion result can be used to achieve semantic control and feature reorganization in complex, multi-element scenes.

[0049] In one optional embodiment, firstly, using a multimodal alignment module, first image features extracted from multiple first images are used as key-value pairs as input, and text features extracted from the target descriptive text are used as query vectors. A cross-attention mechanism is then used to calculate the semantic relevance weights between the text features and each image feature. Next, the first image features are weighted and aggregated based on these semantic relevance weights to achieve semantically selective fusion of multiple visual primitives. Finally, the aggregated features undergo multi-layer nonlinear transformation and feature normalization to generate a joint feature representation semantically aligned with the target descriptive text, which serves as the final fusion result. This process, through text-guided dynamic feature selection and fusion, avoids redundant interference from multiple image information, improves the fine-grainedness and consistency of semantic-visual alignment, provides accurate, stable, and interpretable conditional inputs for subsequent generation models, and enhances the controllability and accuracy of image generation in complex scenarios.

[0050] In one optional embodiment, firstly, using a multimodal alignment module, the first image features extracted from multiple first images are input in parallel and uniformly mapped to a shared semantic space through a learnable projection layer. Simultaneously, the text features extracted from the target description text are segmented at the word or phrase level to form multi-granularity semantic units. Then, a cross-modal gating fusion mechanism is employed to perform element-wise gating modulation on the features of each first image based on the text features, filtering out visual components strongly correlated with the text semantics. Finally, the fusion result is generated through feature concatenation and weighted summation. This approach achieves semantically driven visual feature selection through a differentiable gating structure, reducing computational complexity, improving the real-time performance and stability of feature fusion, while maintaining accurate responses to multi-element semantics, effectively enhancing the fusion result's performance in semantic integrity and structural consistency.

[0051] Step S108: Based on the fusion result and the target description text, generate the target image corresponding to the image generation request.

[0052] The aforementioned target image can refer to high-fidelity visual content that conforms to semantic instructions and is output based on the common conditions driven by the fusion result and the target description text. The target image can be used to fully restore the scene elements and visual relationships defined in the target description text, providing synthetic training samples with strong realism, high annotation accuracy, and wide scene coverage for autonomous driving data augmentation.

[0053] In one optional embodiment, firstly, the fusion result is used as a visual conditional input to the diffusion generation model. Simultaneously, the target description text is encoded as semantic constraints, and the image distribution is iteratively optimized in the latent space through a conditional sampling process. This ensures that the generated content follows the semantic structure of the fusion result and the semantic constraints of the target description text in terms of composition, lighting, target attributes, and spatial relationships. Finally, the target image corresponding to the image generation request that matches the input conditions is output. This process, through semantic-visual joint conditional control, improves the ability of the generated image to reproduce complex, multi-element scenes and enhances structural accuracy. It effectively avoids the problems of missing elements, semantic conflicts, and detail distortion that occur under traditional single-text or single-image guidance, providing a realistic and semantically complete synthetic data foundation for subsequent high-precision automatic annotation and model training.

[0054] In one alternative embodiment, the fusion result can be injected into the latent space of the generative model as an initial visual prior. Simultaneously, the target description text is mapped to conditional embeddings via an encoder. Both are then jointly input into a diffusion-based generative network through channel-level concatenation or feature modulation. This guides the model to reconstruct semantically correct visual content pixel-by-pixel during denoising, ensuring that the generated target image maintains semantic consistency with the fused features and text description in terms of target category, pose, location, and environmental interaction. This approach, through a joint conditionally guided diffusion mechanism, achieves fine-grained semantic control while maintaining the generative model structure. It effectively enhances the realism of multi-target coexistence, occlusion relationships, and environmental consistency in complex scenes, reduces semantic bias and visual artifacts in generated samples, and provides highly reliable synthetic image support for automated data augmentation.

[0055] This application proposes an image generation method. First, based on the target description text in the image generation request, multiple semantic elements in a driving scene are determined. Next, multiple first images matching one of the semantic elements are retrieved from a preset database. Then, a multimodal alignment module is used to fuse the first image features extracted from the multiple first images with the text features extracted from the target description text to generate a fusion result. Finally, based on the fusion result and the target description text, the target image corresponding to the image generation request is generated. This application first decomposes the driving scene into multiple independently matchable semantic elements based on the target description text, and retrieves the first images corresponding to each semantic element from the preset database as the generation basis. Then, by using the first image features of the multiple first images as structural priors and fusing them with text features, the generation process avoids the semantic vagueness and structural conjecture caused by relying solely on abstract text descriptions. Instead, it relies on visual patterns in real data for constrained reconstruction, achieving the technical objective of improving the consistency between the generated image and the real driving scene in terms of target shape, spatial relationship, and scene consistency. This achieves a dual enhancement in semantic accuracy and visual realism of the generated image, thus solving the technical problem of low image generation accuracy in related technologies.

[0056] Optionally, the multimodal alignment module is used to fuse the first image features extracted from multiple first images and the text features extracted from the target description text to generate a fusion result, including: performing positional encoding on the text features to obtain encoded features; performing self-attention processing on the encoded features to obtain attention features; and performing cross-attention processing on the first image features and attention features to generate a fusion result.

[0057] The aforementioned positional encoding refers to applying a learnable vector related to positional information to each word in the text feature sequence extracted from the target descriptive text, thus embedding sequence order information into the semantic representation. Positional encoding methods can include, but are not limited to, sinusoidal positional encoding and learnable embedded positional encoding; the specific method needs to be determined based on actual needs. Positional encoding can compensate for the lack of positional awareness in text features themselves, enabling the model to distinguish the sequential relationship of semantic elements in the description (e.g., "the mining truck in the front left" versus "the mining truck is in the front left"), thereby improving the accuracy of semantic structure representation.

[0058] The aforementioned encoding features can refer to a sequence of text feature vectors enhanced by positional encoding. The types of encoding features can include, but are not limited to, word-level encoding features and phrase-level encoding features; the specific encoding features need to be determined based on the encoding granularity. Encoding features can be used to carry both semantic content and syntactic structure information, providing context-aware input for subsequent self-attention mechanisms and supporting accurate modeling of multi-word semantic combinations.

[0059] The aforementioned self-attention processing refers to calculating pairwise correlations between elements within the encoded features, dynamically learning the dependencies between terms within the text through a query, key, and value mechanism. Self-attention processing methods can include, but are not limited to, single-head self-attention and multi-head self-attention; the specific method must be determined based on actual needs. Self-attention processing can be used to model implicit semantic relationships in text descriptions, enhancing the completeness and consistency of semantic expression and eliminating redundant or conflicting descriptions.

[0060] The aforementioned attention features can refer to the weighted and aggregated sequence of text features output after self-attention processing, reflecting the high-order semantic representation formed after semantic interactions within the text. Attention features can be used as intermediate representations after semantic refinement to guide the direction of visual feature fusion, ensuring that subsequent cross-attention focuses only on semantically enhanced key semantic components, rather than the original lexical noise.

[0061] The aforementioned cross-attention processing refers to using attention features as the query and first image features as the key and value, calculating the response strength of text semantics to each image feature through attention weights, thus achieving semantic-driven weighted fusion of visual features. Types of cross-attention processing can include, but are not limited to, unidirectional cross-attention and bidirectional cross-attention; the specific cross-attention processing method needs to be determined based on actual needs. Cross-attention processing can be used to establish fine-grained alignment relationships between text semantics and multiple visual primitives, ensuring that the generated fusion result retains only visual components strongly related to the description, suppressing irrelevant background or interfering targets.

[0062] In one optional embodiment, firstly, the text features are positionally encoded, embedding sequence order information into the original text representation to form encoded features. Next, a self-attention mechanism is used to perform internal semantic interaction on the encoded features, mining contextual dependencies between lexical units to generate attention features focused on semantic associations. Subsequently, using the attention features as queries, cross-attention calculations are performed with visual features extracted from multiple first images, dynamically weighting and filtering the visual components most relevant to the semantic description, and aggregating them into a unified fusion result. This process, through position-aware enhancement of the text sequence and adaptive semantic extraction, achieves semantically guided filtering and structured fusion of visual features, effectively improving the alignment accuracy between multi-image information and complex text instructions, suppressing irrelevant visual noise interference, and providing highly consistent and semantically faithful conditional inputs for subsequent image generation.

[0063] In one optional embodiment, multiple retrieved images (e.g., image 1 representing "beard", image 2 representing "strong light", and image 3 representing "wearing a safety helmet") and complex text descriptions are input into the "alignment head" module. The core of this module is a cross-attention mechanism: Key / Value: features from multiple images; Query: features from the complex text description. The working principle is to use text as guidance, through multiple cross-attention calculations, to "filter" and "extract" the key features most relevant to the text description from multiple images (e.g., extracting the "beard" feature from image 1 and the "strong light" feature from image 2), and finally fusing them into a unified feature representation rich in multi-feature information.

[0064] Optionally, the multimodal alignment module is obtained by training the initial alignment module using first training data, which includes: sample images, multiple sample elements, and sample features of the sample images, wherein the multiple sample elements are obtained by semantic parsing of the sample images; or, the multimodal alignment module is obtained by jointly training the initial alignment module and the image generation module using second training data, which includes: training samples, sample description text corresponding to the training samples, and the image generation module is used to generate a target image based on the fusion result and the target description text.

[0065] The aforementioned first training data can refer to the supervised dataset used to train the initial alignment module separately. The first training data may include, but is not limited to, sample images, multiple sample elements obtained through semantic parsing of the sample images, and sample features of the sample images. The specific first training data needs to be determined based on actual requirements. The first training data can be used to provide a supervisory signal for the "image-semantic" correspondence, enabling the alignment module to learn how to extract and fuse visual content consistent with the semantic description from multiple image features.

[0066] The aforementioned sample images can refer to the original visual input in the first training data, which can be a single or multiple images with clear semantic labels. The types of sample images can include, but are not limited to, single-feature sample images and multi-feature sample images; the specific sample images need to be determined according to actual needs. Sample images can be used as a source of visual feature extraction, providing visual primitives of the real scene for the alignment module, and used to construct the semantic mapping relationship between images and text.

[0067] The aforementioned sample elements can refer to structured descriptive units extracted after semantic parsing of the sample image, corresponding to identifiable environmental, target, attribute, or behavioral components in the image. These sample elements may include, but are not limited to, environmental elements, target elements, attribute elements, and behavioral elements; the specific sample elements need to be determined based on the actual sample image. These sample elements can be used as semantic supervision signals to guide the alignment module in learning the correlation between different visual features and semantic components, achieving fine-grained cross-modal alignment.

[0068] The aforementioned sample features can refer to the high-dimensional vector representation extracted from the sample image by the encoder. Sample features can be used to reflect the visual semantic content of the image. Sample features may include, but are not limited to, global sample features and local sample features; the specific sample features need to be determined according to actual needs. Sample features can be used as input to the visual side for matching training with sample elements, enabling the alignment module to establish a mapping channel from visual representation to semantic description.

[0069] The aforementioned second training data can refer to the supervised dataset used for jointly training the initial alignment module and the image generation module. The second training data may include, but is not limited to, training samples and corresponding sample description texts; the specific second training data needs to be determined based on actual requirements. The second training data can be used to provide end-to-end generation objectives, enabling the alignment module and generation module to collaboratively optimize and ensure that the fusion result effectively drives the image generation module to output high-quality images that conform to semantics.

[0070] The aforementioned initial alignment module can refer to an untrained multimodal alignment network architecture. The initial alignment module can be used as a core component for feature fusion, independently improving semantic-visual alignment capabilities on the first training data, or co-evolving with the generation module on the second training data.

[0071] The aforementioned image generation module can refer to an image synthesis unit. The image generation module may include, but is not limited to, diffusion models and generative models; the specific image generation module needs to be determined based on actual requirements. The input to the image generation module is the fusion result and the target description text, and the output is the target image. The image generation module can be used to take the fusion features output by the alignment module as visual conditions, combined with textual semantic constraints, to generate a high-fidelity image that conforms to semantic instructions.

[0072] The aforementioned training sample can refer to a single data unit in the second training data. Training samples can include, but are not limited to, single-image training samples and multi-image training samples; the specific training samples need to be determined based on actual needs. Training samples can be used as the minimum supervised unit for joint training, driving the alignment module and the generation module to jointly approximate the distribution and semantic consistency of the real image.

[0073] The aforementioned sample description text can refer to a structured natural language description of the image content in the training samples, aligned with the image semantics. This sample description text can be used as semantic instructions for the generation module and a supervision target for the alignment module, ensuring that the generated images are semantically aligned with the target image. Figure 1 This will improve the effectiveness of annotation and training.

[0074] In one optional embodiment, a first training data comprising sample images, multiple sample elements obtained by semantic parsing of the sample images, and sample features corresponding to the sample images can be used to supervise the training of the initial alignment module, so that the module learns to accurately extract and aggregate representations aligned with semantic elements from the visual features of multiple images, thereby obtaining a multimodal alignment module.

[0075] Alternatively, a second training data, including training samples and corresponding sample description text, can be used to perform end-to-end joint training on the initial alignment module and the image generation module. This allows the output fusion result of the multimodal alignment module to directly drive the generation module to produce an image consistent with the semantic description, thus obtaining the multimodal alignment module.

[0076] The above mechanism achieves independent enhancement of alignment capabilities and generation-alignment co-evolution through separate training and joint training modes, respectively. Without relying on a large amount of real data, it effectively improves the semantic alignment accuracy of multiple image features and complex text descriptions, and enhances the structural integrity and feature accuracy of generated images, providing stable, controllable and iterative high-quality conditional inputs for data augmentation in long-tail scenarios.

[0077] Optionally, obtaining multiple first images from a preset database includes: matching each semantic element among multiple semantic elements with the semantic tags of each image in the preset database to obtain an image set corresponding to the semantic element, wherein the semantic tags of the images are obtained by semantic parsing the images; and obtaining multiple first images based on the image set corresponding to multiple semantic elements.

[0078] The aforementioned semantic tags refer to structured descriptive fields generated after semantic parsing of an image. Semantic tags can include, but are not limited to, environment tags, target tags, attribute tags, and behavior tags, with the specific semantic tags determined based on the actual image. Semantic tags can be used as a searchable semantic index for image content, matching it with input semantic elements to filter visual samples corresponding to specific semantic descriptions from the database.

[0079] The aforementioned image set can refer to a collection of all images that satisfy a certain semantic condition, selected after matching a semantic element with the semantic labels of images in a pre-defined database. The image set can include, but is not limited to, single-element image sets and multi-element intersection image sets; the specific image set needs to be determined based on matching requirements and matching granularity. The image set can be used to provide a visual reference sample pool for each semantic element, used to subsequently construct multiple first images, ensuring that the visual features fused during the generation process have real-world scene support and semantic consistency.

[0080] In one optional embodiment, firstly, each semantic element from multiple semantic elements is matched with the semantic tag of each image in a preset database. For each semantic element, all images whose semantic tags contain that element are selected, forming an independent image set. Then, at least one image is selected from each image set and combined to form multiple first images, ensuring that each first image corresponds to a visual instance that can be clearly represented by the semantic element. This process, through semantic tag-driven retrieval, achieves a reproducible mapping from structured semantic descriptions to real visual samples, ensuring that the selected first images are consistent with the semantic elements in terms of environment, target, attribute, or behavior dimensions. This effectively improves the semantic relevance and scene realism of subsequent multi-image feature fusion, providing a visually reliable foundational input for controllable image generation.

[0081] Optionally, based on the fusion result and the target description text, a target image corresponding to the image generation request is generated, including: using the fusion result as a control condition to guide the image generation model to generate the target image based on the target description text.

[0082] The aforementioned control conditions can refer to the fusion result output by the multimodal alignment module, serving as a structured visual guidance signal for the image generation model. These control conditions can constrain the latent spatial sampling direction of the image generation model during the image generation process, ensuring that the generated content follows the scene priors implied by the fusion result in terms of spatial structure and visual element distribution. Simultaneously, they work in conjunction with the semantic instructions of the target descriptive text to prevent the generated result from deviating from the real scene structure or exhibiting issues such as element misalignment or missing elements.

[0083] In one optional embodiment, firstly, the fusion result is input into the image generation model as a control condition, and fused at the feature level with the semantic features extracted from the target description text by a text encoder to form a joint condition vector. Then, guided by this joint condition vector, the image generation model gradually denoises through a diffusion process in the latent space. Each noise prediction step is simultaneously constrained by visual priors such as the compositional structure, target relative position, and ambient lighting implied in the fusion result, as well as semantic elements in the target description text. Ultimately, a target image is reconstructed that is consistent with the spatial structure of the fusion result and whose semantic content matches the target description text. This generation process achieves synergistic constraints between visual priors and semantic instructions through feature-level conditional coupling, avoiding structural distortion or element misalignment caused by simple text guidance. It also eliminates the need for additional image editing or spatial control networks. Without altering the generation model architecture, it improves the scene rationality, element accuracy, and engineering usability of the generated image, providing a reliable technical path for the high-quality, highly consistent automated generation of long-tail scene data for autonomous driving.

[0084] Optionally, the method further includes: determining a reference image based on an image generation request; obtaining a second image matching the reference image from a preset database; and fusing first image features extracted from multiple first images and text features extracted from target description text using a multimodal alignment module to generate a fusion result, including: fusing first image features, second image features extracted from the second image, and text features using a multimodal alignment module to generate a fusion result.

[0085] The aforementioned reference image can refer to the initial visual input specified by the image generation request to trigger database retrieval. Reference images may include, but are not limited to, manually selected scene baseline images, original sample images reported by the vehicle, etc., and the specific reference image needs to be determined based on the actual situation. The reference image can be used as a benchmark for semantic and visual similarity matching, driving the preset database to retrieve a second image with a similar scene, providing realistic background and environmental context support for the generation process.

[0086] The aforementioned second image can refer to one or more images retrieved from a preset database by matching the reference image in visual content or semantic structure, i.e., the result of image search. The second image can be used as a visual supplement to the environmental context, working in conjunction with the features of the first image to provide scene-based information (such as lighting conditions, terrain features, and background structure), enhancing the multimodal alignment module's perception of global composition and environmental consistency, and making the fusion result closer to the visual distribution of real driving scenarios.

[0087] The aforementioned second image features refer to the high-dimensional visual feature vector obtained after feature extraction from the second image. These second image features can reflect the global scene structure, environmental background, and spatial layout information carried by the second image. The types of second image features may include, but are not limited to, environmental type features, lighting condition features, and weather condition features; the specific second image features need to be determined based on the second image. The second image features can provide a realistic scene contextual prior for the multimodal alignment module, assisting in constructing the background composition, lighting distribution, and environmental consistency of the generated image. This makes the subsequent fusion results more spatially similar to real driving scenarios, avoiding background distortion or scene voids caused by relying solely on target object features (first image features).

[0088] In one optional embodiment, after determining the reference image based on the image generation request, the multimodal feature similarity between the reference image and each image in the preset database is calculated. Images with high semantic-visual consistency with the reference image in terms of scene composition, lighting conditions, environment type, or target layout are selected as second images. This matching process uses distance measurement based on feature vectors extracted by a pre-trained encoder to select the most similar single or a small number of images. This step, through accurate retrieval of real-world scene images, provides engineering-credible environmental context support for subsequent feature fusion, effectively enhancing the visual reproduction capability of complex and rare driving scenarios during the generation process. It ensures that the generated content closely resembles the real data distribution in terms of background structure and environmental features, while avoiding reliance on simulation or generalized models with insufficient generalization ability, thus improving the controllability and reliability of the entire generation process.

[0089] In one optional embodiment, a multimodal alignment module is used to jointly align first image features, second image features, and text features through a cross-attention mechanism. The first image features represent the local semantics and morphology of the target object, the second image features provide the global structure and environmental priors of the scene background, and the text features constrain the semantic intent of the generated content. The module uses text features as a query to dynamically and weightedly aggregate relevant visual information from the first and second image features, outputting a unified high-dimensional feature representation that integrates target attributes, environmental context, and semantic instructions as the fusion result. This process enables fine-grained semantic collaboration between multi-source visual information and text instructions, improves the modeling accuracy of the fusion result for the coexistence of multiple elements in complex long-tail scenes, effectively suppresses feature conflicts and semantic biases, and provides conditional guidance for subsequent image generation that combines structural realism and semantic accuracy.

[0090] Optionally, the method further includes: acquiring a set of driving data uploaded by multiple vehicles, wherein the driving data set is uploaded by multiple vehicles when abnormal events in the driving environment are detected during driving; acquiring multiple preset images from the driving data set, wherein the perceptual confidence of the multiple preset images is less than a preset confidence level, and / or the recognition accuracy is less than a preset accuracy level; performing semantic parsing on the multiple preset images based on at least one semantic dimension to generate semantic labels for the multiple preset images, wherein the at least one semantic dimension includes at least one of the following: environmental variables of the driving environment, type, attributes, and behavior of targets in the driving environment; and updating a preset database based on the multiple preset images and the semantic labels of the multiple preset images.

[0091] The aforementioned driving dataset refers to the raw data automatically uploaded by the vehicle's onboard perception system based on preset trigger conditions during the driving process of multiple vehicles. The driving dataset may include, but is not limited to, image frames, radar point clouds, vehicle status (such as speed and steering angle), timestamps, and raw detection results output by the perception model. The specific driving dataset needs to be determined based on the actual situation. The driving dataset can be used as the raw data source for long-tail and challenging scenarios, to identify instances where the model performs poorly in real-world driving environments, and to support the automated mining of subsequent abnormal samples and database construction.

[0092] The aforementioned abnormal events refer to events determined by the vehicle perception system based on a rule engine or model confidence mechanism that deviate significantly from normal driving behavior or expected perception output. Abnormal events can be used as the sole condition for triggering the upload of driving data, ensuring that the collected data is concentrated in scenarios where the model fails or the environment is extreme, and avoiding redundant data interference.

[0093] The aforementioned multiple preset images can refer to single-frame images selected from the driving data set that meet the conditions of a perception confidence level lower than a preset confidence level and / or a recognition accuracy level lower than a preset accuracy level. The types of multiple preset images may include, but are not limited to, missed detection images (the target is clearly visible but the model does not output a detection box), false detection images (non-target objects are incorrectly identified as target categories), positioning deviation images (the position, size, or orientation of the detection box deviates significantly from the true target), and images affected by severe weather (such as rain, fog, strong light, or dust causing the target to be blurred or distorted). The specific preset images need to be determined based on the actual situation. Multiple preset images can be used as direct visual evidence of model perception failure, to build a high-value sample library, and to support subsequent semantic parsing and database updates.

[0094] The perceived confidence level mentioned above can refer to the confidence score of a preset image. Perceived confidence level can be used as a standard to quantify model uncertainty, automatically filter low-confidence predictions, and locate the model's vulnerabilities in specific scenarios. In this application, perceived confidence level can be used to determine whether to trigger data upload or filter preset images.

[0095] The aforementioned preset reliability can refer to a pre-defined threshold used to filter out low-confidence perceived results. The preset reliability can be used as a quantitative standard to determine whether an image belongs to the "preset image".

[0096] The aforementioned recognition accuracy refers to the geometric matching degree between the target detection box in the preset image and the real target. Recognition accuracy can be used to supplement the confidence index, identifying low-quality detections that have output but inaccurate localization, thus avoiding missed detections due to relying solely on confidence. Recognition accuracy can be represented by the Intersection over Union (IoU) value, ranging from [0,1]. When the IoU is less than the preset accuracy, it is considered insufficient.

[0097] The aforementioned preset accuracy rate refers to a preset accuracy threshold used to determine whether the detection box localization is acceptable. The preset accuracy rate can serve as a quantitative basis for filtering out inaccurately identified images. The preset accuracy rate can work in conjunction with a preset reliability level to form a dual screening condition, improving the purity of "difficult samples" in the preset images.

[0098] The aforementioned at least one semantic dimension can be a semantic classification dimension of the preset image content. At least one semantic dimension may include, but is not limited to, environmental variables of the driving environment, the type of target in the driving environment, the target's attributes, and the target's behavior, etc. The specific semantic dimension needs to be determined based on the content of the preset image. At least one semantic dimension can be used to transform unstructured images into standardized semantic descriptions, realize a searchable mapping between images and text, and support intelligent database organization and prompt word generation.

[0099] The aforementioned semantic tags can refer to structured text descriptions automatically generated after comparing and analyzing a preset image and its original perceptual output based on at least one semantic dimension. Semantic tags may include, but are not limited to, basic prompt words and combinations of structured phrases; the specific semantic tags need to be determined based on the preset image.

[0100] In one optional embodiment, firstly, a set of driving data uploaded by multiple vehicles during driving due to the detection of abnormal events in the driving environment is acquired; then, based on perception confidence and / or recognition accuracy, multiple preset images with perception confidence lower than a preset confidence level and / or recognition accuracy lower than a preset accuracy level are selected from the driving data set; furthermore, based on at least one semantic dimension including environmental variables of the driving environment, target type, target attributes, and target behavior, semantic parsing is performed on the multiple preset images to generate structured semantic tags corresponding to the multiple preset images; finally, the preset database is updated according to the mapping relationship between the multiple preset images and the semantic tags of the multiple preset images.

[0101] The above process automatically captures model failure samples through vehicle-side anomaly triggering mechanisms, and ensures the high value of samples by combining confidence and positioning accuracy dual screening. It utilizes predefined semantic dimensions to achieve standardized automatic annotation of image content, forming searchable and reusable structured data assets. This provides real, accurate, and scenario-rich basic sample support for subsequent image generation, improves the coverage accuracy and engineering usability of generated data for long-tail and difficult driving scenarios, and builds an efficient, closed-loop, and low-manual-dependency data augmentation foundation.

[0102] Optionally, multiple preset images are acquired from the driving data set, including: acquiring multiple preset images from multiple original images based on one or a combination of perceptual confidence, consistency verification results, event correlation, anomaly detection results, and manual feedback results from multiple original images in the driving data set; wherein, perceptual confidence is used to characterize the confidence of multiple vehicles in perceiving each original image; consistency verification results are obtained by verifying the temporal consistency of each original image using a first verification model, and by comparing the output of the first verification model with the output of at least one second verification model, the output of the first verification model being obtained by verifying the original image using the first verification model, and the output of at least one second verification model being obtained by verifying the original image using at least one second verification model; event correlation is obtained based on the time deviation between the acquisition time of each original image and the occurrence time of the abnormal event; anomaly detection results are obtained by detecting anomalies based on the image features of each original image; and manual feedback results are obtained by the target object verifying the original image after each original image is output to the target object.

[0103] The aforementioned consistency verification result can refer to a comprehensive judgment result calculated based on the temporal continuity of the original image and the consistency of multi-model outputs. The consistency verification result may include a temporal consistency score, which is a continuity score calculated by the first verification model to determine whether the target trajectory, size, and category in consecutive frames conform to the laws of physical motion (e.g., abrupt shifts in inter-frame displacement, target disappearance-reappearance anomalies). The consistency verification result may also include a multi-model consistency difference value, which is a difference measure calculated by comparing the detection outputs of the first verification model with those of at least one second verification model. The consistency verification result can be used to identify false anomalies caused by sensor noise, transient interference, or model fluctuations, eliminate non-robust samples, and improve the accuracy and stability of preset image selection.

[0104] The aforementioned first verification model can refer to a verification model used for real-time, continuous analysis of the temporal sequence of the original image. Its input consists of multiple consecutive frames of images and their perceived output, and the output is a temporal rationality score. The first verification model can include, but is not limited to, a Kalman filter based on trajectory prediction, a motion consistency analysis model based on optical flow, or a temporal behavior modeling model, etc. The specific first verification model needs to be determined according to actual needs. The first verification model can be used to identify abnormal frames that do not conform to physical laws from a temporal dimension, such as instantaneous target jumps, trajectory breaks, or unnatural acceleration, as a basis for filtering false anomaly samples.

[0105] The aforementioned at least one second verification model can refer to an auxiliary verification model that runs in parallel with the first verification model and has a different architecture or training data. At least one second verification model can be used to perform independent perceptual verification on the same original image. The second verification model can be a shadow model, i.e., an offline high-precision model running in the cloud. Each model outputs independent detection results, which are used for cross-validation with the first verification model. By identifying the differences in outputs among at least one second verification model, systematic biases or misjudgments by a single model can be identified, improving the robustness of anomaly detection and avoiding incorrect screening due to defects in a single model.

[0106] The aforementioned event correlation refers to the time difference between the original image acquisition time and the time of the known abnormal event, used to measure the spatiotemporal relevance of an image to a key driving event. Event correlation can be used to focus on image segments directly related to real dangerous behaviors (such as emergency braking and collision warning), improve the semantic value of samples, and avoid acquiring irrelevant background frames.

[0107] The aforementioned anomaly detection results can refer to an anomaly score calculated using unsupervised or weakly supervised anomaly detection algorithms based on the visual feature vectors of the original image. These results reflect the degree to which the original image deviates from its normal distribution in the feature space. Furthermore, the anomaly detection results can automatically identify potential "unknown anomaly" samples that are not labeled by the perceptual model but exhibit abnormal visual distributions at the high-dimensional feature level, supplementing the blind spots of perceptual confidence-based screening.

[0108] The aforementioned human feedback results refer to the confirmation or correction results given by human annotators or domain experts after receiving the original image, based on visual judgment, to determine whether the image is a valid reference image. Human feedback results can be used as a final verification step to correct mis-screening or omissions in the automated process, thereby improving the semantic accuracy and annotation quality of the preset image set.

[0109] In one optional embodiment, starting from multiple original images in the driving dataset, the perception confidence is first determined based on the confidence score attached to the detection results output by the vehicle perception model for each frame. Then, a first verification model is used to perform temporal consistency analysis on the continuous image sequence, calculating the physical plausibility score of the target trajectory, size, and category between frames. This result is compared with the detection results independently output by at least one second verification model for the same image to obtain a consistency verification result, reflecting the stability of the judgments between models. Simultaneously, based on the time deviation between the timestamp of each original image and the trigger time of a known abnormal event (such as emergency braking, collision warning, or driver takeover), the event correlation is calculated, retaining only images within a preset time window before and after the event. Then, using the visual feature vectors extracted from the original images, an unsupervised anomaly detection algorithm (such as autoencoder reconstruction error or feature space outlier) is used to calculate the anomaly detection result, identifying potential anomalies that are not labeled by the perception model but whose feature distribution significantly deviates from normal samples. Finally, the automatically selected original images are pushed to a manual annotation platform, where the target object performs manual verification based on visual judgment, generating manual feedback results, including confirmation of validity, exclusion of invalidity, or correction of semantic information. Finally, by comprehensively considering one or more of the following factors—perceived confidence level, consistency verification results, event correlation, anomaly detection results, and human feedback results—multiple preset images that meet the criteria for high-value anomalies are selected.

[0110] The above process uses a multi-source, heterogeneous, and hierarchical judgment mechanism to identify low-confidence, high-semantic-difficulty samples of model failure in real-world scenarios, effectively filtering noise and pseudo-anomalies, improving the accuracy, representativeness, and engineering usability of preset images, and providing a high-quality, low-human-intervention data source for subsequent structured semantic annotation and database construction, supporting the automated and closed-loop enhancement of long-tail scenario data for autonomous driving.

[0111] Optionally, the method further includes: generating evaluation prompts for the target image based on at least one evaluation metric corresponding to the target image, wherein different evaluation metrics are used to characterize the conditions for evaluating the generation quality of the target image from different dimensions; inputting the target image and the evaluation prompts into an image evaluation model, using the image evaluation model to evaluate the target image, and obtaining a target evaluation result for the target image, wherein the target evaluation result is used to characterize whether the target image meets at least one evaluation metric; determining the target type of the target image based on the target evaluation result; and adjusting the image generation model based on the target adjustment strategy corresponding to the target type; wherein, in the case of the target type being the first type, the target adjustment strategy is to train the image generation model using the target image; in the case of the target type being the second type, the target adjustment strategy is to analyze the defects of the image generation model using the target image, obtain the analysis results, and adjust the image generation model based on the analysis results; and in the case of the target type being the third type, the target adjustment strategy is to manually review the target image, obtain the manual review results, and adjust the image generation model using the target image based on the manual review results.

[0112] At least one of the aforementioned evaluation metrics can refer to standardized conditions used to quantitatively evaluate the quality of the generated target image. Each metric measures the degree to which the generated result conforms to the expected semantic or visual requirements from a specific dimension. At least one evaluation metric may include, but is not limited to, mean Average Precision (mAP), recall, cue word compliance, object completeness, attribute binding accuracy, spatial relationship correctness, scene reproduction, realism, and logical rationality. The specific at least one evaluation metric needs to be determined based on actual evaluation requirements. At least one evaluation metric can be used as the basis for constructing evaluation cue words, clarifying the measurable dimensions of image quality, and supporting the objectivity and consistency of the automated evaluation process.

[0113] The aforementioned evaluation prompts can refer to structured natural language instructions derived from at least one evaluation metric corresponding to the target image. Evaluation prompts can guide image evaluation models to make systematic judgments about images. They can also be used to transform abstract evaluation metrics into standardized instructions that can be understood by a Vision-Language Model (VLM), enabling automated and reproducible image quality evaluation.

[0114] The aforementioned image evaluation model can refer to an automated evaluation engine built upon a Visual-Language Large Model (VLM). This model can replace manual initial screening, enabling efficient, consistent, and quantifiable quality assessment of large-scale generated images, thus automating and scalable the evaluation process.

[0115] The aforementioned target evaluation result can refer to the structured judgment result output by the image evaluation model after analyzing the target image based on evaluation prompts. The target evaluation result can be used as direct input for target type classification, determining the trigger path for subsequent adjustment strategies.

[0116] The aforementioned target type refers to the image category categorized based on the target evaluation results, used to guide subsequent differentiated processing strategies for the image generation model. Target types may include, but are not limited to, Type 1, Type 2, and Type 3, with the specific type determined based on the target evaluation results. Target types can be used to achieve closed-loop management of evaluation and feedback, automatically classifying images according to their quality levels and triggering corresponding adjustment paths.

[0117] The first type mentioned above can refer to image categories where the target image meets at least one evaluation metric. The first type represents images of acceptable quality and can be directly used as high-quality data for model training. It can also provide positive enhancement samples for image generation models, allowing for fine-tuning to improve overall generation capability and generalization.

[0118] The second type mentioned above can refer to image categories where the target image partially meets or does not meet at least one evaluation metric. This second type indicates that the generative model has systematic defects in specific dimensions (such as attribute binding, occlusion handling, and lighting restoration). This second type can be used to pinpoint the weaknesses of the generative model, supporting targeted adjustments and avoiding blind training.

[0119] The above analysis results can refer to the defect patterns of the generative model in terms of semantic understanding, feature fusion, or conditional control, summarized based on the evaluation results of the second type of images. The analysis results can be used to provide actionable adjustment directions for adjusting the image generation model, realizing a closed loop of problem identification, strategy formulation, and model correction.

[0120] The third type mentioned above refers to image categories where the target evaluation result is ambiguous or borderline, and the image evaluation model cannot clearly determine whether they are qualified. This third type may include, but is not limited to, semantic ambiguity, near-realistic conditions, and conflicts among multiple indicators; the specific third type needs to be determined based on the target evaluation result. This third type can be used to identify blind spots in the model's evaluation capabilities, introducing human intervention to improve the robustness and accuracy of the evaluation system.

[0121] The aforementioned manual review results may refer to the final confirmation conclusion given by domain experts after visually judging the target image of the third type, which serves as the basis for arbitration of the uncertainty of model evaluation.

[0122] In one optional embodiment, firstly, based on at least one evaluation metric corresponding to the target image, structured evaluation prompts are generated for the target image. These prompts are converted from conditions representing different dimensions of generation quality (such as prompt compliance, object integrity, attribute binding accuracy, spatial relationship correctness, etc.) into natural language instructions. Next, the target image and evaluation prompts are input into an image evaluation model. The model performs semantic alignment and logical reasoning on the image content based on the prompts, outputting a target evaluation result to determine whether the image meets the listed evaluation metrics. Finally, based on the discrete state of the target evaluation result (satisfied, partially satisfied, not satisfied), the target type of the target image is determined, completing the automated grading of the generated target image quality. This process, through the structured mapping of evaluation metrics and prompts, achieves multi-dimensional, quantifiable, and standardized automatic evaluation of generation quality, improving evaluation efficiency and consistency. It provides reliable input for the accurate classification and differentiated adjustment of the subsequent image generation model, supporting the efficient operation of a high-quality data closed loop.

[0123] In one optional embodiment, the image generation model is differentiated based on the target adjustment strategy corresponding to the target type. Specifically, when the target type is type one, the target image is directly used for training the image generation model to enhance its ability to generate scenes that meet the evaluation criteria. When the target type is type two, a systematic analysis is performed on the generation defects exposed in the target image (such as attribute mismatch, spatial relationship errors, or environmental distortion) to obtain a structured description of the defect patterns, and the parameters or control conditions of the image generation model are adjusted accordingly for targeted repair. When the target type is type three, manual review is introduced to determine its final quality label, and the image is used as a correction sample in model training based on the review results to alleviate the problem of ambiguous judgment boundaries in the evaluation model. This hierarchical adjustment mechanism achieves precise matching of the generation model adjustment path, avoids ineffective training and resource waste, improves model iteration efficiency and generalization ability, and enhances the robustness of the evaluation system through a closed-loop manual review, forming a closed-loop optimization mechanism from quality judgment to model correction.

[0124] Optionally, the method further includes: when the target type is the first type, performing multi-level annotation on the target image to obtain target annotation results for the driving scene, wherein different levels of annotation processes result in different levels of accuracy in annotating the target image; adjusting the model parameters of the vehicle perception model based on the target image and the target annotation results to obtain an adjusted perception model; testing the adjusted perception model to obtain test results for the adjusted perception model; and deploying the adjusted perception model on at least one vehicle based on the test results; wherein performing multi-level annotation on the target image to obtain target annotation results for the driving scene includes: using a target detection model to annotate the target image... Note: Initial annotation results are obtained; based on the initial annotation results, the target segmentation model is guided to annotate the target image to obtain target annotation results, wherein the accuracy of the target annotation results is greater than the accuracy of the initial annotation results; based on the test results, the adjusted perception model is deployed on at least one vehicle, including: deploying the adjusted perception model on multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or deploying the adjusted perception model on some of the multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or running the adjusted perception model on multiple vehicles, and the output of the adjusted perception model is not coupled with the control link of the multiple vehicles.

[0125] The aforementioned multi-level annotation refers to an automated annotation process that performs target image annotation in stages and with varying levels of precision. Each level of annotation improves semantic or geometric precision based on the previous level. Multi-level annotation may include, but is not limited to, primary annotation, i.e., bounding box annotations output by the target detection model, which have low precision and only locate the approximate area of ​​the target; and advanced annotation, i.e., pixel-level mask annotations generated by the target segmentation model based on the primary annotation, which fit the target contour and have higher precision than the primary annotation. Multi-level annotation can be used to gradually improve the accuracy of annotation results while ensuring annotation efficiency, achieving a progressive improvement from coarse localization to pixel-level precise annotation, adapting to the needs of large-scale data processing.

[0126] The aforementioned vehicle perception model can refer to the original model deployed on an autonomous vehicle for real-time perception of the surrounding environment and output of target detection, classification, and tracking results. The vehicle perception model can be used as an input module for the vehicle decision-making system, responsible for identifying and locating targets (such as vehicles, pedestrians, and obstacles) in the road environment.

[0127] The aforementioned adjusted perception model can refer to an improved version of the vehicle perception model obtained by updating or fine-tuning the parameters based on the target image and its high-precision target annotation results. The adjusted perception model can be used to improve the model's perception ability in long-tailed and difficult scenes (such as nighttime, occlusion, and low contrast), and enhance its generalization and robustness in real-world environments.

[0128] The test results mentioned above refer to the quantitative performance metrics output after evaluating the adjusted perception model on an independent test set. These results can be used to determine whether the adjusted perception model meets the deployment requirements. They can also serve as a basis for deciding whether the model can be deployed online, ensuring that model iteration does not lead to overall performance degradation.

[0129] The aforementioned object detection model refers to a visual model used to identify object categories from input images and output their bounding box positions. Object detection models can include, but are not limited to, anchor-based detection models, single-stage detection models, or text-guided zero-shot detection models; the specific object detection model needs to be determined based on actual needs. Object detection models can be used to achieve fast, coarse-grained localization of object images, providing initial spatial guidance for subsequent high-precision annotation, reducing segmentation computational complexity, and improving overall annotation efficiency.

[0130] The initial annotation results mentioned above refer to the target location and category information generated by the target detection model in the form of bounding boxes, which is the first stage output of multi-level annotation. The initial annotation results can be used as the starting point of the multi-level annotation process, providing a rough spatial range of the target to guide subsequent segmentation models to focus on the effective region and reduce interference from irrelevant backgrounds.

[0131] The aforementioned target segmentation model refers to a model that performs pixel-level semantic segmentation of a target image based on spatial guidance provided by initial annotation results. Target segmentation models may include, but are not limited to, instance segmentation models, two-stage models based on mask prediction, or conditional segmentation networks, etc. The specific target segmentation model needs to be determined according to actual needs. Target segmentation models can be used to convert coarse bounding box annotations into pixel-level precise annotations, improving the fit and annotation quality of target contours, and providing high-fidelity data for perceptual model training.

[0132] The aforementioned target annotation results can refer to pixel-level accurate target contour masks generated by the target segmentation model based on the initial annotation results. Target annotation results can be used as high-precision ground truth for fine-tuning vehicle perception models, improving their localization accuracy and boundary adaptability on complex and irregular targets (such as occluded pedestrians or falling rocks on slopes).

[0133] In one optional embodiment, when the target type is a first type, multi-level annotation is first performed on the target image to obtain target annotation results for the driving scene. The multi-level annotation process includes two progressively more precise levels. First, the image is identified using a target detection model, and an initial annotation result containing the target category and a rectangular bounding box is output. Then, using this initial annotation result as spatial guidance, it is input into a target segmentation model to generate a pixel-level mask that fits the true contour of the target, serving as a more precise target annotation result. Next, the target image and its corresponding target annotation result are used together to fine-tune the parameters of the vehicle perception model, resulting in an adjusted perception model. Subsequently, the performance of the adjusted perception model is evaluated on an independent test set to obtain test results with recall rate and detection rate of specific long-tail scenes as the core. Finally, based on whether the test results meet a preset performance threshold, it is decided to deploy the adjusted perception model on at least one vehicle. The above process improves annotation accuracy through a two-level annotation mechanism to ensure high-quality training data; enhances the model's ability to perceive key scenarios through precise parameter adjustment; and achieves safe and controllable model updates through a layered deployment strategy, avoiding system risks caused by model degradation, thus forming a closed-loop adjustment chain from high-precision annotation to safe deployment.

[0134] In one optional embodiment, the adjusted perception model is deployed on at least one vehicle based on the test results. This includes deploying the adjusted perception model on all vehicles, replacing the original vehicle perception models, and achieving a unified system-level upgrade. It also includes deploying the adjusted perception model only on some vehicles, replacing the original perception models of the corresponding vehicles, to verify the model's stability and performance within a limited scope. Furthermore, it includes running the adjusted perception model on all or some vehicles, but its output is not connected to the vehicle's control decision-making chain; it is only collected and compared as by-the-loop observation data and does not interfere with the vehicle's actual behavior. These three deployment methods constitute a progressive deployment strategy from full replacement, batch pilot testing to parallel shadow mode. This strategy can verify the model's reliability in real driving environments in stages while ensuring system safety, effectively avoiding driving risks caused by model performance fluctuations, and providing real-world scenario feedback for model iteration, achieving safe and controllable continuous adjustments.

[0135] In one alternative embodiment, Figure 2 This is an execution flowchart of an image generation system according to an embodiment of this application, such as... Figure 2 As shown, the method includes:

[0136] Step S201, data mining.

[0137] Step S202: Feature fusion generation.

[0138] Step S203: Generate closed-loop data based on the fusion of diffusion model and scene library.

[0139] Step S204: Automatic evaluation and filtering of the quality of generated data based on multi-dimensional quantitative indicators.

[0140] Step S205: High-precision automatic annotation based on the fusion of prompt words and instance segmentation.

[0141] Step S206: Iterative training and validation of the vehicle-side perception model based on augmented data.

[0142] Specifically, for step S201, data mining, it includes:

[0143] SS1, data reporting and collection, monitors the driving environment in real time through the vehicle-side perception system. When potential anomalies are detected, it automatically reports relevant data (images, radar point clouds, timestamps, vehicle status, etc.) to the cloud data platform. The reporting mechanism is divided into proactive reporting (detection of abnormal events) and passive collection (periodic sampling) to ensure data coverage of various driving scenarios.

[0144] SS2, or small sample data mining, involves extracting "hard samples" and "long-tail samples" from massive amounts of real-world driving data using multi-dimensional filtering strategies. It utilizes natural language prompts to guide a visual-language model (VLM) to intelligently analyze unlabeled data, automatically generating candidate anomaly samples and corresponding scene descriptions.

[0145] Specifically, small sample data mining can include the following five stages:

[0146] Phase 1 involves real-time vehicle-side judgment based on the confidence threshold and rule engine of the vehicle perception model. The confidence threshold and rules may include, but are not limited to, the model output confidence being lower than the preset confidence threshold, overlapping and conflicting detection boxes (IOU anomaly), and sudden or unreasonable target trajectory changes.

[0147] Phase Two involves offline cloud-based analysis based on multi-model voting and consistency verification, including: comparison of results between the Shadow Model and the online model, analysis of differences in results between different versions of the model, and temporal consistency verification (detection of sudden changes between frames).

[0148] Phase 3 involves event-driven triggering based on correlation analysis of abnormal vehicle events. The data used may include, but is not limited to, data from 3 seconds before and after an emergency braking event, data from the moment a collision warning is triggered, and data from the moment a driver takes over or performs an abnormal operation.

[0149] Phase four involves feature space analysis based on embedding anomaly detection. The anomaly detection methods mentioned above may include, but are not limited to, calculating the difference between image feature vectors and database feature distributions, performing feature space clustering, identifying outliers, and using autoencoder methods to identify feature reconstruction error anomalies.

[0150] Phase 5, based on expert review and manual final verification by the annotation platform, pushes the automatically selected candidate samples to the annotation platform, where annotators confirm the anomaly type and correct the annotations. Finally, after expert review, the samples are archived to the corresponding BadCase (abnormal sample) library.

[0151] For the small number of key samples selected, data augmentation is performed using a generative model to expand sample diversity and construct a high-quality small sample training set.

[0152] SS3, intelligent analysis and basic prompt word generation.

[0153] We selected a high-performance Visual-Language Model (VLM) as the base model to perform in-depth analysis on candidate samples in the database.

[0154] First, the identification labels of the vehicle-side model are obtained from the reported data. Then, the identification labels of the vehicle-side model are compared with those of the VLM (Vehicle Modeling Library) to obtain the following error types: anomaly type (missed detection, false detection, positioning deviation, etc.), scene features (weather, lighting, road type, etc.), and target attributes (category, size, occlusion, etc.). Next, the generated prompt words are matched one-to-one with the sample images to form structured annotations, facilitating subsequent retrieval and model training. Finally, the prompt words are obtained from the VLM.

[0155] Example of a prompt word sample: Generating contextualized Bad Case prompt words.

[0156] Typical bad case scenarios and keyword extraction examples: The system automatically extracts scenario keywords from abnormal samples to form a structured scenario description, facilitating classification, retrieval, and targeted adjustments, including:

[0157] For scenarios where perception misses detection, the description is that the target is clearly visible in the image, but the perception model fails to detect it. Keyword extraction includes: sandstorm weather + small excavator + blurred edges + low contrast; nighttime + pedestrians without reflective markings + low light + motion blur; heavy rain + motorcycle + water mist obscuring + specular reflection.

[0158] For perceptual false detection scenarios, the description is that the model misidentifies non-target objects as targets, or phantom detection occurs. Keyword extraction includes: tree shadow + misidentified as pedestrian + lighting projection + similar texture; billboard figure + misidentified as real person + planar image + perspective distortion; bridge structure + misidentified as obstacle + complex geometry + distant small target.

[0159] For scenarios involving positioning deviation, the description is that the detection box position, size, or orientation is inaccurate. Keyword extraction includes: large mining truck + partial occlusion + bounding box offset + perspective distortion; large vehicle at close range + bounding box overflowing the image boundary + truncation processing; oblique vehicle + orientation estimation error + non-horizontal bounding box.

[0160] For severe weather scenarios, the description is a decrease in perception performance under extreme weather conditions. Keyword extraction includes: dense fog + visibility <50 meters + target edge blurring + extremely low contrast; sandstorm + color distortion + uniform texture + lack of detail; strong backlight + lens flare + overexposure of highlights + shadow areas.

[0161] For special event scenarios, the description is an uncommon but important driving scenario. Keyword extraction includes: construction area + temporary traffic cones + non-standard placement + reflective material; accident scene + multiple vehicles stacked + irregular shape + emergency vehicle lights; animal crossing + sudden intrusion + rapid movement + non-rigid deformation.

[0162] Therefore, we can identify and process abnormal bad cases in small samples and their basic keywords.

[0163] For step S202, feature fusion generation includes:

[0164] ss1, builds a basic prompt word and image library.

[0165] Mining areas have complex and variable environments (such as sandstorms, fog, and rugged terrain). General text-based image models cannot generate accurate scene images that meet engineering requirements based solely on prompts, resulting in uncontrollable generation results, missing elements, or distortion.

[0166] To address this issue, this application proposes constructing a structured database of basic prompt words and images. The core of this database is establishing a one-to-one mapping between "prompt words" and "basic images." Before inputting the data into the generative model, the system finds the corresponding basic image for each prompt word. It first matches the most suitable basic image as visual guidance based on the instructions, then combines this with the prompt word to feed the model into generating corresponding high-fidelity, highly controllable images for specific scenes.

[0167] Further subdivisions are as follows: Environmental Variables: Describe the macro-level background of the scene. Basic descriptive terms: Time (late at night), Location (edge ​​of the open-pit mine), Weather (thunderstorm), Light (dim, only vehicle headlights provide illumination). Mining area-specific environment: Sandstorms, dense fog, heat rising from equipment, pervasive dust. Target Type: Define the core object or subject requiring attention. Key Equipment: Large mining trucks, hydraulic excavators, bulldozers. Key Elements: Falling rocks on slopes, deep potholes on roads, pedestrians violating traffic rules, drivers. Target Attributes: Describe the state characteristics of the target. Equipment State: Fully loaded, unloaded, damaged, stationary, in motion. Personnel State: Driver (fatigued, not wearing a seatbelt), safety officer. Target Behavior: Define the actions the target is performing. Vehicle Behavior: Sharp turns, overtaking, reversing, convoy driving. Personnel Behavior: Making phone calls, smoking, leaning out of the vehicle, illegally crossing the road.

[0168] This database is not a simple graph atlas, but rather a set of "visual-semantic" anchors tailored to the generative model. The construction process includes:

[0169] Data Acquisition and Structured Annotation: Extensive collection of real-world mining area scene images, video frames, and high-fidelity simulation images. A complete set of structured descriptive text is assigned to each image, generated based on a predefined multi-dimensional tagging system. This system includes at least:

[0170] Environmental variables include time (e.g., late at night), location (e.g., the edge of an open-pit mine), weather (e.g., thunderstorms), light (e.g., dim lighting), and the unique environment of the mining area (e.g., sandstorms, dense fog).

[0171] Target / task variables include target type (e.g., large mining truck, rockfall on slope), target attributes (e.g., fully loaded, damaged), and target behavior (e.g., overtaking, making a phone call).

[0172] Data import and index construction: A one-to-one mapping relationship is established between the processed images and their corresponding structured text descriptions, and the data is stored in the database. Simultaneously, a multimodal encoder is used to extract feature vectors from all images and texts, constructing an efficient vector retrieval index that supports text-to-image, image-to-image, and text-to-text searches.

[0173] ss2 is a multi-image feature guidance and fusion module that can be trained separately.

[0174] The main improvement of the separable, trainable multi-image feature guidance and fusion module lies in the addition of a customizable and trainable "guidance and fusion" module to the front end of the generative model. The core process of this customizable and trainable "guidance and fusion" module is as follows:

[0175] A domain-specific image and text database is constructed, in which each image is labeled with detailed tags, mainly divided into categories such as environment, target, and behavior, which constitute the foundation of the system's knowledge.

[0176] sss1, multimodal retrieval:

[0177] When a user inputs a badcase image (an image that needs improvement) and a complex text description, the system first performs a search, proceeding as follows: Image search: Searching the database for images similar to the badcase image to serve as the basis for generation. Text search: Based on multiple "basic clue words" in the complex text description (such as "beard," "strong light," "wearing a safety helmet"), retrieving from the database the set of images that best represent each single feature.

[0178] SSS2, text-guided multi-image feature fusion:

[0179] Input the retrieved images (e.g., image 1 represents "beard", image 2 represents "strong light", image 3 represents "wearing a safety helmet") and complex text descriptions into the "Align Head" module.

[0180] The core of this module is a cross-attention mechanism: Key / Value: features from multiple images; Query: features from complex text descriptions.

[0181] The working principle is to use text as a guide, and through multiple cross-attention calculations, to "filter" and "extract" the key features most relevant to the text description from multiple images (for example, extract the "beard" feature from image 1 and the "strong light" feature from image 2), and finally merge them into a unified feature representation rich in multi-feature information.

[0182] The separable training strategy includes: Independent training: The "alignment head" module can be separated from the large-scale generative model at the backend and trained independently. Training data: "Complex scene images" from a self-built domain database are used as ground truth. Complex images are decomposed into multiple basic prompts, and corresponding simple images are retrieved. Then, the ground truth features of the complex images are used to supervise the output features of the fused "alignment head". Loss function: Contrastive learning loss (such as cosine similarity loss function) is used to make the fused features closer to the features of the ground truth images. Joint training: When the domain data accumulates to a preset amount, this module can also be jointly trained end-to-end with the generative model, and further adjustments can be made using human quality control feedback.

[0183] For step S203, the closed-loop data generation based on the fusion of the diffusion model and the scene library includes:

[0184] S2031, Scene Instruction Parsing and Basic Image Matching: The system receives natural language generation instructions and automatically parses and extracts environmental variables (e.g., nighttime, rain / fog, curves) and target types (e.g., large mining trucks, small vehicles). Subsequently, in a pre-built basic image library containing multiple scenes, the system retrieves and matches the basic environment image that best matches the environmental variables.

[0185] S2032, Conditional Controlled Generation: The matched base environment image is used as a strong visual condition input, and together with the fully structured cue words constructed based on instructions, it is input into the diffusion model. The model uses the composition and environment of the aforementioned base image as a foundation to generate precise content at the specified location that conforms to the target attributes (e.g., full load) and target behavior (e.g., overtaking). For dynamic or rare behavioral targets, image editing or controllable generation techniques such as ControlNet can be used for fusion generation.

[0186] S2032, Output and System Iteration: The model outputs the final generated scene image. This image can be automatically added to the base image library to adjust subsequent matching effects, forming a self-reinforcing data generation loop.

[0187] For step S204, automatic assessment and filtering of the generated data quality based on multi-dimensional quantitative indicators includes:

[0188] Since the inherent mechanism of generative models can lead to problems such as image structural distortion, blurred details, and incorrect attribute binding, this application also proposes an automated quality assessment and filtering mechanism based on a large visual language model.

[0189] S2041, Construct a multi-dimensional quality assessment system: Establish a quantitative assessment system that includes objective and subjective dimensions. The objective dimension focuses on the adherence to prompts, covering object completeness, attribute binding accuracy, spatial relationship correctness, and scene reproduction accuracy. The subjective dimension assesses realism, logical rationality, and aesthetic quality. Weights are assigned to different dimensions; for example, in autonomous driving data generation tasks, prompt adherence and realism are given higher weights.

[0190] S2042, Design and execute automated VLM assessment: Based on the aforementioned assessment system, design structured expert-level assessment prompts to guide a VLM base model in performing a systematic quality assessment of batch-generated images. The assessment prompts clearly define roles, assessment dimensions, processes, and standardized output formats (such as JSON) to ensure the objectivity and consistency of the assessment.

[0191] S2043, Automatic Data Classification and Manual Review: Based on the comprehensive quantitative score given by the VLM model, the generated data is automatically classified into three categories: **Golden Data:** High-quality samples, directly input into the database for model training. **Negative Sample Data:** Low-quality samples, automatically filtered or used to analyze defects in the generated model. **Difficult-to-Define Data:** Borderline samples, sent for manual review and final judgment.

[0192] S2044, Evaluation-Generation Dual-Model Iterative Optimization: Accurately labeled data generated through a closed-loop manual review process is used to train and optimize the VLM evaluation model, improving its evaluation capabilities. Simultaneously, the classified prime data, negative sample data, and corrected boundary data are used for positive reinforcement training, defect-specific adjustments, and generalization capability improvement of the diffusion generation model, respectively, forming a complete optimization closed loop of co-evolution between data generation and quality evaluation.

[0193] For step S205, high-precision automatic annotation based on the fusion of prompt words and instance segmentation includes:

[0194] S2051, Problem identification: The bounding boxes generated by the automatic annotation system based on the generative model are usually coarse (e.g., the default is to generate square bounding boxes), which cannot closely fit the true contour of irregular targets, resulting in insufficient annotation accuracy.

[0195] S2052, High-Precision Automatic Annotation Process: To improve annotation accuracy, a two-stage automatic annotation process is constructed: Coarse Localization: Using a zero-shot detection model based on cue words, an initial bounding box of the target is generated based on the text description. Fine Segmentation: Guided by the initial bounding box, it is input into an instance segmentation model (such as the Segment Anything Model, or SAM) to obtain a pixel-level segmentation mask of the target. Post-processing Transformation: The pixel-level segmentation mask is converted into a minimum bounding box that closely fits the target contour, serving as the final high-precision annotation.

[0196] S2053, Adjustment of annotation prompts: Introduce detailed descriptions of the target's location, shape, and orientation in the generated annotation prompts to guide the generation or detection model to produce an initial localization that is more consistent with spatial logic, providing a better starting point for subsequent segmentation steps.

[0197] S2054, Long-tail data augmentation application combines the above-mentioned high-precision automatic annotation process with the data generation method described in step S203 to achieve automatic and accurate annotation of data generated in long-tail scenarios such as "mining truck overtaking" and "nighttime rockfall", completing end-to-end automation from data generation to annotation and expanding high-quality training samples.

[0198] For step S206, the iterative training and validation of the vehicle-side perception model based on augmented data includes:

[0199] S2061, Model Training and Open-Loop Testing. The vehicle-side perception model is supplemented (fine-tuned) using the "golden data" selected through quality assessment in step S204 and the automatically labeled "long-tail data" in S5. After training, performance is evaluated on an open-loop test set containing known bad cases, with a focus on how well the original bad cases are handled.

[0200] S2062, Evaluation and Decision on the Golden Test Set. The model is tested on an independent, full-scenario golden test set. Key performance indicators (such as mAP and recall) are compared before and after adjustments. If key indicators show significant deterioration, the model version is rolled back, the cause of the problem is analyzed, and the process returns to step S203 or S205 to adjust the data generation or labeling strategy. If the indicators remain stable or improve, and open-loop testing shows that the target anomalous cases have been resolved, proceed to the next step.

[0201] S2063, Phased Deployment and Closed-Loop Validation: Models that pass evaluation enter the phased deployment process: Shadow Mode Validation: The model runs in "shadow mode" on real vehicles. Its output is not coupled to the vehicle control link and is only used to collect data and compare it with the online model results to further verify stability. Small-Scale Batch Deployment: The original model is gradually replaced on a selection of vehicles, and online metrics and Bad Case reporting rates are continuously monitored. Full Deployment: The model iteration is finally completed, and new cases are continuously collected through the vehicle-side data reporting system (step S201), forming a complete closed loop from data mining, generation, evaluation to model training and deployment.

[0202] In one alternative embodiment, Figure 3 This is a flowchart of a fusion processing method according to an embodiment of this application, such as... Figure 3 As shown, firstly, based on the reference image and basic prompt words, multiple database images are determined from the multimodal database, namely database image 1, database image 2, ..., database image n, where the aforementioned reference image is also the bad case image; next, the multiple database images are combined with the same reference image and input into the visual transformer for processing to obtain image features; then, the image features are vectorized and encoded to obtain multiple image tags.

[0203] Simultaneously, the complete prompt words are vectorized and encoded to obtain text tags. Then, the text tags are processed using a large language backbone model and input into a learnable multimodal alignment head. The aforementioned multiple image tags are also input into the learnable multimodal alignment head, which fuses the text tags with the multiple image tags to obtain fused feature tags. Finally, a diffusion model is used to process the fused feature tags to obtain the generated image.

[0204] Specifically, firstly, the abnormal sample image (i.e., the Bad Case image) reported by the vehicle is used as a reference image. Combined with the structured scene prompt words automatically generated by the vision-language big data model, multiple basic images (denoted as database image 1, database image 2, ..., database image n) that best match the semantics of the prompt words are retrieved from a pre-built domain-specific multimodal database. Then, the reference image and the above multiple basic images are input into the VisionTransformer (ViT) for encoding, and the local semantic features of each image are extracted to form the corresponding image feature sequence. After linear projection processing, a set of structured image tokens is obtained. Simultaneously, the complete structured prompt words are input into the text encoder to generate corresponding text tokens. These text tokens are further semantically enhanced and contextually modeled by the Large Language Model (LLM) backbone, outputting high-order semantic representations. Subsequently, the enhanced text tokens and multiple image tokens are simultaneously input into a learnable multimodal alignment head. This alignment head employs a cross-attention mechanism, using text semantics as the query and image tokens as the key / value pair. Through multi-head attention weighted fusion, it dynamically filters and aggregates the visual elements most relevant to the prompt words, generating unified fusion feature tokens. Finally, these fusion feature tokens, as strong visual conditions, are input together with the original structured prompt words into a diffusion model. Using the composition and lighting of the reference image as a basis, conditional denoising is performed to generate a high-fidelity synthetic image that conforms to the real engineering scene.

[0205] The above process, through a collaborative mechanism of reference image guidance, multi-base graph retrieval, text semantic driving, and alignment head fusion, achieves high-precision and highly controllable image generation for complex, long-tailed, and multi-element autonomous driving scenarios.

[0206] Figure 4 This is an optional image generation flowchart according to an embodiment of this application, used to implement multi-image guided semantic fusion and feature alignment, such as... Figure 4 As shown, the method includes:

[0207] First, multiple base images (denoted as image 1, image 2, ..., image n) retrieved from the multimodal database are input into the visual transformer for encoding. Each image is divided into multiple image patches, which are then processed by linear embedding and Transformer layers to output a set of corresponding local visual features, collectively referred to as the image feature sequence. This sequence is further used to form a two-dimensional feature map (n×dim), where n is the number of image patches and dim is the feature dimension. This two-dimensional feature map serves as a key and value for use in the subsequent multimodal alignment head.

[0208] At the same time, the system receives structured scene prompts (generated by the LLM backbone), inputs them into the text encoder to generate a text query vector (word query), and injects query position embedding based on the order and role of each semantic component in the prompts to form a structured query sequence as the query input.

[0209] Subsequently, the text query vector (Q) and feature maps (K, V) from multiple base images are input into a learnable multimodal alignment head. This module includes a multi-head cross-attention mechanism, where the query comes from the text and the key / value comes from image features. Through multi-head parallel computation, the visual components most relevant to the text semantics are dynamically weighted and aggregated, outputting a fused intermediate feature sequence. This intermediate feature is processed by layer normalization and residual connections (Add) before being input into a feedforward neural network (FFN) for nonlinear transformation and semantic enhancement, outputting a high-dimensional semantic fusion feature. The fused feature is further compressed and semantically aligned by a projection alignment module, mapped to a target feature vector (1×dima). This vector is a unified semantic representation of the complex scene jointly expressed by the input multiple images and text prompts.

[0210] Meanwhile, to achieve supervised learning of the alignment head, the system uses the ground truth image (gt image) corresponding to the above multi-image input, extracts its global features through CLIP or ViT encoder, and uses it as the ground truth target feature (gt feature, 1×dima); at the same time, the ground truth text (gt word) corresponding to the ground truth image is used to generate the text query vector for supervision to ensure semantic consistency.

[0211] During training, the parameters of the projection alignment module are adjusted in reverse by calculating the cosine similarity loss between the predicted target features (1×dima) and the ground target features (1×dima), so that the fused features approximate the semantic distribution of the real complex scene in the feature space, thereby achieving "small sample, high precision" cross-modal alignment.

[0212] In one alternative embodiment, Figure 5 This is a flowchart illustrating an image generation process in a mining area scene according to an embodiment of this application. The process includes:

[0213] Step S502: Based on the missed images reported by the vehicle-side perception system, determine them as reference images.

[0214] Step S504: In the preset mining area image and text database, the second image that is most semantically similar to the reference image is retrieved by embedding vector matching.

[0215] Step S506: Extract the first image features of the reference image and the second image features of the second image, respectively.

[0216] Step S508: Extract the text features corresponding to the target description text;

[0217] Step S510: Input the first image features, the second image features, and the text features into the multimodal alignment module, and generate fused features through cross-attention fusion.

[0218] Step S512: Using the fusion features as a condition, drive the diffusion model to generate high-fidelity images and automatically complete pixel-level annotations, which are then stored in the database for model training.

[0219] The above process, guided by the collaborative interaction of reference and matching images, improves the structural integrity and semantic accuracy of generated images in occluded and missing scenes.

[0220] According to an embodiment of this application, an embodiment of an image generation apparatus is provided. It should be noted that the apparatus can be used to perform the image generation method described above.

[0221] Figure 6 This is a schematic diagram of an image generation apparatus according to an embodiment of this application, such as... Figure 6 As shown, the device includes: a determining module 602, an acquiring module 604, a fusing module 606, and a generating module 608.

[0222] The determination module 602 is used to determine multiple semantic elements in the driving scene based on the target description text in the image generation request; the acquisition module 604 is used to acquire multiple first images from a preset database, wherein each of the multiple first images is matched with one of the multiple semantic elements; the fusion module 606 is used to fuse the first image features extracted from the multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate a fusion result; and the generation module 608 is used to generate the target image corresponding to the image generation request based on the fusion result and the target description text.

[0223] Optionally, the fusion module is used to perform positional encoding on the text features to obtain encoded features; to perform self-attention processing on the encoded features to obtain attention features; and to perform cross-attention processing on the first image features and the attention features to generate a fusion result.

[0224] Optionally, the multimodal alignment module is obtained by training the initial alignment module using first training data, which includes: sample images, multiple sample elements, and sample features of the sample images, wherein the multiple sample elements are obtained by semantic parsing of the sample images; or, the multimodal alignment module is obtained by jointly training the initial alignment module and the image generation module using second training data, which includes: training samples, sample description text corresponding to the training samples, and the image generation module is used to generate a target image based on the fusion result and the target description text.

[0225] Optionally, the acquisition module is used to match each semantic element among multiple semantic elements with the semantic tags of each image in a preset database to obtain the image set corresponding to the semantic element, wherein the semantic tags of the images are obtained by semantic parsing the images; based on the image set corresponding to multiple semantic elements, multiple first images are obtained.

[0226] Optionally, the generation module is used to use the fusion result as a control condition to guide the image generation model to generate the target image based on the target description text.

[0227] Optionally, the device is also used to determine a reference image based on an image generation request; obtain a second image that matches the reference image from a preset database; and fuse the first image features extracted from multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate a fusion result, including: fusing the first image features, the second image features extracted from the second image, and the text features using a multimodal alignment module to generate a fusion result.

[0228] Optionally, the device is further configured to acquire a set of driving data uploaded by multiple vehicles, wherein the driving data set is uploaded by multiple vehicles when abnormal events in the driving environment are detected during driving; acquire multiple preset images from the driving data set, wherein the perceptual confidence of the multiple preset images is less than a preset confidence level, and / or the recognition accuracy is less than a preset accuracy level; perform semantic parsing on the multiple preset images based on at least one semantic dimension to generate semantic labels for the multiple preset images, wherein the at least one semantic dimension includes at least one of the following: environmental variables of the driving environment, type, attributes and behaviors of targets in the driving environment; and update the preset database based on the multiple preset images and the semantic labels of the multiple preset images.

[0229] Optionally, the device is further configured to acquire multiple preset images from multiple original images based on one or a combination of perceptual confidence, consistency verification results, event correlation, anomaly detection results, and human feedback results from multiple original images in a driving dataset; wherein, perceptual confidence is used to characterize the confidence of multiple vehicles in perceiving each original image; consistency verification results are obtained by verifying the temporal consistency of each original image using a first verification model, and by comparing the output of the first verification model with the output of at least one second verification model, wherein the output of the first verification model is obtained by verifying the original image using the first verification model, and the output of at least one second verification model is obtained by verifying the original image using at least one second verification model; event correlation is obtained based on the time deviation between the acquisition time of each original image and the occurrence time of the abnormal event; anomaly detection results are obtained by detecting anomalies based on the image features of each original image; and human feedback results are obtained by the target object verifying the original image after each original image is output to the target object.

[0230] Optionally, the device is further configured to generate evaluation prompts for the target image based on at least one evaluation metric corresponding to the target image, wherein different evaluation metrics are used to characterize the conditions for evaluating the generation quality of the target image from different dimensions; input the target image and evaluation prompts into an image evaluation model, use the image evaluation model to evaluate the target image, and obtain a target evaluation result for the target image, wherein the target evaluation result is used to characterize whether the target image meets at least one evaluation metric; determine the target type of the target image based on the target evaluation result; and adjust the image generation model based on the target adjustment strategy corresponding to the target type; wherein, when the target type is the first type, the target adjustment strategy is to train the image generation model using the target image; when the target type is the second type, the target adjustment strategy is to analyze the defects of the image generation model using the target image, obtain the analysis result, and adjust the image generation model based on the analysis result; and when the target type is the third type, the target adjustment strategy is to manually review the target image, obtain the manual review result, and adjust the image generation model using the target image based on the manual review result.

[0231] Optionally, the device is further configured to perform multi-level annotation on the target image when the target type is the first type, to obtain target annotation results for the driving scene, wherein different levels of annotation processes result in different levels of accuracy in annotating the target image; adjust the model parameters of the vehicle perception model based on the target image and the target annotation results, to obtain an adjusted perception model; test the adjusted perception model to obtain test results of the adjusted perception model; and deploy the adjusted perception model on at least one vehicle based on the test results; wherein performing multi-level annotation on the target image to obtain target annotation results for the driving scene includes: annotating the target image using a target detection model. The initial annotation results are obtained; based on the initial annotation results, the target segmentation model is guided to annotate the target image to obtain the target annotation results, wherein the accuracy of the target annotation results is greater than the accuracy of the initial annotation results; based on the test results, the adjusted perception model is deployed on at least one vehicle, including: deploying the adjusted perception model on multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or deploying the adjusted perception model on some of the multiple vehicles and replacing the vehicle perception model with the adjusted perception model; or running the adjusted perception model on multiple vehicles, and the output of the adjusted perception model is not coupled with the control link of the multiple vehicles.

[0232] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0233] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0234] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0235] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0236] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0237] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0238] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0239] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0240] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0241] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0242] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image generation method, characterized in that, include: Based on the target description text in the image generation request, multiple semantic elements in the driving scene are identified. Multiple first images are obtained from a preset database, wherein each of the multiple first images is matched with one of the multiple semantic elements; The first image features extracted from the multiple first images and the text features extracted from the target description text are fused using a multimodal alignment module to generate a fusion result; Based on the fusion result and the target description text, the target image corresponding to the image generation request is generated.

2. The method according to claim 1, characterized in that, The process of fusing the first image features extracted from the multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate the fusion result includes: The text features are positionally encoded to obtain encoded features; The encoded features are subjected to self-attention processing to obtain attention features; The first image features and the attention features are subjected to cross-attention processing to generate the fusion result.

3. The method according to claim 2, characterized in that, The multimodal alignment module is obtained by training an initial alignment module using first training data. The first training data includes: sample images, multiple sample elements, and sample features of the sample images. The multiple sample elements are obtained by semantic parsing of the sample images; or... The multimodal alignment module is obtained by jointly training the initial alignment module and the image generation module using second training data. The second training data includes: training samples and sample description text corresponding to the training samples. The image generation module is used to generate the target image based on the fusion result and the target description text.

4. The method according to claim 1, characterized in that, Retrieve multiple first images from a pre-defined database, including: Each semantic element among the plurality of semantic elements is matched with the semantic tags of each image in the preset database to obtain the image set corresponding to the semantic element, wherein the semantic tags of the images are obtained by semantic parsing the images; Based on the image set corresponding to the multiple semantic elements, the multiple first images are obtained.

5. The method according to claim 1, characterized in that, The step of generating the target image corresponding to the image generation request based on the fusion result and the target description text includes: The fusion result is used as a control condition to guide the image generation model to generate the target image based on the target description text.

6. The method according to any one of claims 1 to 5, characterized in that, Also includes: Based on the image generation request, a reference image is determined; Obtain a second image that matches the reference image from the preset database; The process of fusing the first image features extracted from the multiple first images and the text features extracted from the target description text using a multimodal alignment module to generate a fusion result includes: The multimodal alignment module is used to fuse the first image features, the second image features extracted from the second image, and the text features to generate the fusion result.

7. The method according to any one of claims 1 to 5, characterized in that, Also includes: Acquire a set of driving data uploaded by multiple vehicles, wherein the driving data set is uploaded by the multiple vehicles when abnormal events in the driving environment are detected during driving; Multiple preset images are obtained from the driving data set, wherein the perception confidence of the multiple preset images is less than the preset confidence, and / or the recognition accuracy is less than the preset accuracy. Semantic parsing is performed on the multiple preset images based on at least one semantic dimension to generate semantic labels for the multiple preset images. The at least one semantic dimension includes at least one of the following: environmental variables of the driving environment, type, attributes and behaviors of targets in the driving environment; The preset database is updated based on the multiple preset images and their semantic tags.

8. The method according to claim 7, characterized in that, The step of acquiring multiple preset images from the driving data set includes: Based on one or a combination of the perceptual confidence, consistency verification result, event correlation, anomaly detection result, and manual feedback result of multiple original images in the driving data set, the multiple preset images are obtained from the multiple original images; The perception confidence level is used to characterize the confidence level of the multiple vehicles in perceiving each original image; The consistency verification result is obtained by verifying the temporal consistency of each original image using the first verification model and by comparing the output result of the first verification model with the output result of at least one second verification model. The output result of the first verification model is obtained by verifying the original image using the first verification model, and the output result of the at least one second verification model is obtained by verifying the original image using the at least one second verification model. The event correlation degree is obtained based on the time deviation between the acquisition time of each original image and the occurrence time of the abnormal event; The anomaly detection results are obtained by performing anomaly detection based on the image features of each original image. The manual feedback result is obtained by the target object verifying the original image after each original image is output to the target object.

9. The method according to any one of claims 1 to 5, characterized in that, Also includes: Based on at least one evaluation index corresponding to the target image, an evaluation prompt word corresponding to the target image is generated, wherein different evaluation indices are used to characterize the conditions for evaluating the generation quality of the target image from different dimensions; The target image and the evaluation prompt words are input into the image evaluation model, and the image evaluation model is used to evaluate the target image to obtain the target evaluation result of the target image, wherein the target evaluation result is used to characterize whether the target image meets the at least one evaluation index; Based on the target evaluation results, the target type of the target image is determined; The image generation model is adjusted based on the target adjustment strategy corresponding to the target type. Wherein, when the target type is the first type, the target adjustment strategy is to train the image generation model using the target image; When the target type is the second type, the target adjustment strategy is to analyze the defects of the image generation model using the target image, obtain the analysis results, and adjust the image generation model based on the analysis results; When the target type is the third type, the target adjustment strategy is to manually review the target image, obtain the manual review result, and adjust the image generation model based on the target image.

10. The method according to claim 9, characterized in that, Also includes: When the target type is the first type, the target image is annotated at multiple levels to obtain the target annotation result of the driving scene. The annotation accuracy of the target image varies depending on the level of annotation. Based on the target image and the target annotation results, the model parameters of the vehicle perception model are adjusted to obtain the adjusted perception model; The adjusted perception model is tested to obtain the test results of the adjusted perception model; Based on the test results, the adjusted perception model is deployed on at least one vehicle; The step of performing multi-level annotation on the target image to obtain the target annotation result of the driving scene includes: The target image is labeled using a target detection model to obtain initial labeling results; Based on the initial annotation results, the target segmentation model is guided to annotate the target image to obtain the target annotation results, wherein the accuracy of the target annotation results is greater than the accuracy of the initial annotation results; The step of deploying the adjusted perception model on at least one vehicle based on the test results includes: The adjusted perception model is deployed on multiple vehicles, and the adjusted perception model replaces the vehicle perception model; or Deploy the adjusted perception model on some of the vehicles among the multiple vehicles, and replace the vehicle perception model with the adjusted perception model; or The adjusted perception model is run on the multiple vehicles, and the output of the adjusted perception model is not coupled to the control links of the multiple vehicles.

11. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 10.

Images