Automatic driving image data generation method and device, equipment and medium

Abstract

The embodiment of the invention provides an automatic driving image data generation method and device, equipment and a medium. The method comprises the following steps: acquiring multi-view image data acquired by a target vehicle in a road scene; performing feature extraction based on the multi-view image data to obtain multiple pieces of first image feature information, and performing fusion processing on the multiple pieces of first image feature information to obtain fused image feature information; generating semantic attributes according to the multi-view image data; inputting the fused image feature information and the semantic attributes into a target model, and outputting three-dimensional scene data; and generating automatic driving image data according to the three-dimensional scene data. According to the embodiment of the invention, the semantic information of key targets such as roads, vehicles and pedestrians is fused into the modeling process, so that the accuracy of target recognition and detail reconstruction is improved, an image conforming to reality is generated, and the generation result can be accurately controlled.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for generating autonomous driving image data. Background Art

[0002] Current image data generation technologies are mainly centered around diffusion models and generative adversarial networks (GANs). Both use random noise as a starting point and generate high-quality images through different mechanisms.

[0003] Diffusion model: Starting from random noise (such as Gaussian noise), a neural network is used to gradually remove the noise. After multiple iterations, a clear image is finally generated.

[0004] GAN model: The generator directly generates images from a random noise vector (usually Gaussian noise or uniformly distributed noise), and optimizes the results through adversarial training with the discriminator.

[0005] However, the above model has the following problems: (1) Generative models (such as diffusion models and GAN models) produce results that are inconsistent with reality when generating images.

[0006] (2) The generation process of the generative model is random and the generation results (such as different perspectives, scenes, etc.) cannot be precisely controlled. Summary of the Invention

[0007] In view of the above problems, a method, apparatus, device, and medium for generating autonomous driving image data are proposed to overcome or at least partially solve the above problems, including: A method for generating autonomous driving image data, the method comprising: Acquire multi-view image data collected by the target vehicle in the road scene; Performing feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generating fused image feature information based on the plurality of first image feature information; generating semantic attributes based on the multi-view image data; Inputting the fused image feature information and the semantic attributes into a target model, and outputting three-dimensional scene data; Automatic driving image data is generated according to the three-dimensional scene data.

[0008] Optionally, generating fused image feature information based on the plurality of first image feature information includes: Determining weight information corresponding to each first image feature information based on a preset dynamic weight mechanism; Performing weighted processing on the weight information and the corresponding first image feature information to obtain second image feature data; The plurality of second image feature data are fused to generate fused image feature information.

[0009] Optionally, generating semantic attributes according to the multi-view image data includes: Inputting the multi-view image data into a CLIP model to extract third image feature data; Semantic attributes of the multi-view image data are generated based on the third image feature data.

[0010] Optionally, inputting the fused image feature information and the semantic attributes into a target model and outputting three-dimensional scene data includes: Acquiring time information corresponding to the multi-view image data; The time information, the fused image feature information, and the semantic attributes are input into a target model, and three-dimensional scene data is output.

[0011] Optionally, generating autonomous driving image data according to the three-dimensional scene data includes: Creating a physical model based on preset climate events, wherein the physical model is used to simulate the interaction between light transmission and the environment to obtain different weather effects; generating simulated weather effects based on the physical model; The simulated weather effect is combined with the three-dimensional scene data to generate autonomous driving image data.

[0012] Optionally, after acquiring multi-view image data collected by the target vehicle in the road scene, the method includes: Preprocessing the multi-view image data includes any one or more of the following: Data labeling processing, image denoising and distortion correction, and data cleaning.

[0013] Optionally, the first image feature information includes any one or more of the following: Image clarity information, viewing angle coverage information, and overlapping area information.

[0014] A device for generating autonomous driving image data, the device comprising: A multi-view image data acquisition module is used to acquire multi-view image data collected by the target vehicle in the road scene; a feature fusion module, configured to perform feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generate fused image feature information based on the plurality of first image feature information; a semantic attribute extraction module, configured to generate semantic attributes based on the multi-view image data; a three-dimensional scene data output module, configured to input the fused image feature information and the semantic attributes into a target model and output three-dimensional scene data; An image data generation module is used to generate autonomous driving image data according to the three-dimensional scene data.

[0015] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method for generating autonomous driving image data as described above.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for generating autonomous driving image data as described above.

[0017] The embodiments of the present invention have the following advantages: In an embodiment of the present invention, multi-view image data captured by a target vehicle in a road scene is acquired. Feature extraction is then performed based on the multi-view image data to obtain multiple first image feature information. Fusion image feature information is then generated based on the multiple first image feature information. Semantic attributes are then generated based on the multi-view image data. The fusion image feature information and semantic attributes are then input into a target model to output three-dimensional scene data. Autonomous driving image data can then be generated based on the three-dimensional scene data. Thus, by introducing semantic attributes into the image, the present embodiment incorporates semantic information about key objects such as roads, vehicles, and pedestrians into the modeling process. Compared to diffusion models, GAN (Generative Adversarial Network) models, or standard NeRF (Neural Radiance Fields) models that only recover information from images, the present embodiment incorporates semantic information into the neural rendering process, thereby improving the accuracy of object recognition and detail reconstruction, generating realistic images, and enabling precise control over the generated results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flowchart of a method for automatically driving image data according to an embodiment of the present invention; Figure 2 This is a flowchart of another method for automatically driving image data according to an embodiment of the present invention; Figure 3 This is a schematic diagram of generating three-dimensional image data provided by one embodiment of the present invention; Figure 4 It is a structural diagram of an apparatus for processing autonomous driving image data provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0020] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0021] Reference Figure 1 , which shows a flowchart of a method for generating autonomous driving image data provided by an embodiment of the present invention, which may specifically include the following steps: Step S101, acquiring multi-view image data collected by a target vehicle in a road scene; In actual applications, an image data acquisition device can be pre-installed in the target vehicle, and then when the vehicle is in a road scene, image data in the road scene can be collected by the image data acquisition device. In order to accurately determine the environment around the target vehicle and realize safe automatic driving of the vehicle, multiple image data acquisition devices can be installed at different positions on the vehicle to collect image data from different perspectives, that is, multi-perspective image data in the embodiment of the present invention.

[0022] Among them, the type of image data acquisition equipment may include but is not limited to using any one or more of a surround-view camera (such as 4 to 8 wide-angle cameras covering a 360° field of view), a front / rear telephoto camera.

[0023] In another embodiment of the present invention, a target vehicle may also be equipped with a laser radar sensor, a millimeter-wave radar sensor, or other sensor data acquisition device to collect the distance between obstacles in the road scene and the target vehicle. This data acquisition device, the laser radar sensor, and the millimeter-wave radar sensor in the target vehicle can then be integrated into a fused multimodal system. Based on the collected distance information and multi-view image data, the surrounding scene can be more accurately analyzed, and the target vehicle can be autonomously controlled based on the surrounding scene to ensure safe driving of the target vehicle.

[0024] In one embodiment of the present invention, to account for differences in collected data due to variations in actual collection scenarios, multiple different collection scenarios can be pre-configured, such as urban roads, highways, and complex traffic scenarios. Each collection scenario must capture data under varying lighting conditions (day and night, backlight), weather conditions (rain, snow, fog), and occlusion. Specific collection settings can be tailored to the specific application of the target vehicle.

[0025] Step S102: performing feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generating fused image feature information based on the plurality of first image feature information; After obtaining multi-view image data, feature extraction can be performed on the multi-view image data to obtain first image feature information that requires special attention in the multi-view image. Specifically, the specific first image feature information to be extracted can be determined based on different usage scenarios, and then a feature extraction method corresponding to the feature information can be determined. The first image feature information can include any one or more of image clarity information, viewing angle coverage information, and overlapping area information.

[0026] Image clarity can be used to measure the clarity of image details, reflecting the image's focus quality or resolution. Image clarity can include key metrics such as resolution, sharpness, and blurriness. Resolution refers to the number of pixels and directly impacts detail rendering. Sharpness refers to edge contrast; in highly sharp images, object edges are more distinct. Blurriness can be quantified using algorithms such as the Laplace gradient.

[0027] View coverage information can be the spatial range captured by a single image or sensor. It can be described by a preset angle (such as horizontal / vertical FOV) or spatial region. View coverage information can be associated with the field of view, shooting distance, and occlusion. In terms of field of view (FOV), a wide-angle lens (such as 120°) covers a wider area than a telephoto lens (such as 30°). The longer the shooting distance, the greater the actual physical range covered. Occlusion may result in incomplete coverage.

[0028] Overlapping area information can be the area that is shared between multiple images and can be expressed as a percentage (e.g., 60% overlap) or as a pixel-matching area. Overlapping area information can be used for feature matching, seamless stitching, and 3D reconstruction of multi-view image data.

[0029] After obtaining multiple first image feature information, the multiple first image feature information can reflect the characteristics of the image from various aspects. Therefore, to ensure the accuracy of the image data, the multiple first image feature information can be combined to obtain fused image feature information. The fusion processing method can be set according to the image requirements in the implementation of the present invention.

[0030] Step S103, generating semantic attributes based on the multi-view image data; In embodiments of the present invention, after acquiring multi-view image data, the multi-view image data can be analyzed to generate semantic attributes. These semantic attributes may include, but are not limited to, semantic attributes associated with objects such as roads, vehicles, and pedestrians. Semantic attributes extract and fuse high-level semantic information (such as object category, material, function, and spatial relationships) from images captured from different viewpoints to build a comprehensive understanding of the scene or object.

[0031] In multi-view images, semantic attributes may include, but are not limited to, any one or more of the following: (1) Object-level attributes: category (such as "car"), material (such as "metal"), color, shape, function (such as "seatable"), etc.

[0032] (2) Scene-level attributes: layout (e.g., “indoor / outdoor”), lighting conditions, season, scene category (e.g., “office”, “street”), etc.

[0033] (3) Relational attributes: relative positions between objects (such as “a cup on the table”), occlusion relationships, size ratios, etc.

[0034] (4) Dynamic attributes (if there is time series data): motion state (such as "walking person"), behavioral intention, etc.

[0035] It should be noted that, in the embodiment of the present invention, step 103 may occur after step 101 captures the multi-view image data.

[0036] Step S104, inputting the fused image feature information and semantic attributes into the target model and outputting three-dimensional scene data; After deriving fused image feature information and semantic attributes from the collected multi-view image data, the fused image feature information can fully utilize the multi-view information, allowing high-quality, highly overlapping views to contribute more significantly to the modeling process. Semantic attributes can capture semantic content, from global structure to local details. By inputting the fused image feature information and semantic attributes into a trained target model for 3D modeling, 3D scene data can be generated from 2D image data.

[0037] In practical applications, the target model can be a pre-trained model for 3D modeling. The input data of this target model is a fusion of image feature information and semantic attributes, and the output data is the 3D scene data corresponding to multi-view images. The target model can be a NeRF (Neural Radiance Fields) model. NeRF is a deep learning-based 3D scene representation and rendering method that can reconstruct high-fidelity 3D scenes from multi-view 2D images and supports novel perspective synthesis. Its core concept is to implicitly model the scene's lighting and geometry through a neural network to achieve realistic rendering effects.

[0038] In one embodiment of the present invention, training data may be prepared in advance to perform iterative training on the selected 3D modeling model to obtain a desired target model, and then the target model may be used to predict 3D scene data.

[0039] Step S105: Generate autonomous driving image data according to the three-dimensional scene data.

[0040] After obtaining the three-dimensional scene data, autonomous driving image data can be generated based on the three-dimensional scene data, and then the image data of the target vehicle appearing in autonomous driving can be displayed.

[0041] In an embodiment of the present invention, the image data may be test image data during the autonomous driving process. A large amount of test image data may be generated by the above method to be applied to various environments for simulating vehicle autonomous driving tests.

[0042] In this embodiment of the present invention, by incorporating semantic attributes into images, semantic information about key objects such as roads, vehicles, and pedestrians can be incorporated into the modeling process. Compared to diffusion models, GAN models, or standard NeRF, which only recover information from images, this embodiment of the present invention incorporates semantic information into the neural rendering process, thereby improving the accuracy of object recognition and detail reconstruction, generating realistic images, and enabling precise control over the generated results.

[0043] Reference Figure 2 , which shows a flowchart of another method for generating autonomous driving image data provided by an embodiment of the present invention, which may specifically include the following steps: Step S201, acquiring multi-view image data collected by a target vehicle in a road scene; Step S202: performing feature extraction based on the multi-view image data to obtain a plurality of first image feature information; Step S203, determining weight information corresponding to each first image feature information based on a preset dynamic weight mechanism; In practical applications, a dynamic weight mechanism may be pre-set, and the dynamic weight mechanism may be used to set a dynamic weight for each first image feature information, and further determine the weight information corresponding to each first image feature information.

[0044] Dynamic weighted fusion does not use fixed weights for all input sources or features, but dynamically adjusts the contribution weights of each part according to the real-time situation or context, so that the fusion results can better adapt to changing environments or data characteristics.

[0045] In one embodiment of the present invention, the dynamic weight mechanism that can be used may be: Confidence-based weighting, performance-based weighting, attention mechanism, online learning adjustment.

[0046] Confidence-based weighting assigns weights based on the output confidence of each input source, such as classification probability or prediction variance. Performance-based weighting adjusts weights based on the recent performance of each model or sensor. Attention mechanisms, such as the self-attention mechanism in the Transformer, can use neural networks to learn dynamic weights. Online learning adjustments can adjust weights based on real-time feedback, using reinforcement learning or online learning algorithms.

[0047] Step S204: performing weighted processing on the weight information and the corresponding first image feature information to obtain second image feature data; After the weight information is determined, each first image feature information may be weighted according to the weight information, that is, the weight information and the corresponding first image feature information are multiplied: second image feature data=weight information*first image feature information.

[0048] Step S205: fusing the plurality of second image feature data to generate fused image feature information.

[0049] After weighted processing, the obtained multiple image feature data can be fused to obtain fused image feature information, so that the modeling accuracy can be improved through dynamic weight distribution, and the obtained fused image feature information can have higher robustness and adaptability.

[0050] Step S206, generating semantic attributes based on the multi-view image data; Step S207, inputting the fused image feature information and semantic attributes into the target model, and outputting three-dimensional scene data; Step S208: Generate autonomous driving image data according to the three-dimensional scene data.

[0051] In one embodiment of the present invention, generating semantic attributes based on multi-view image data includes: inputting the multi-view image data into a CLIP model to extract third image feature data; and generating semantic attributes of the multi-view image data based on the third image feature data.

[0052] In practical applications, a CLIP (Contrastive Language–Image Pretraining) model can be pre-trained. This model takes viewpoint image data as input and outputs third-image feature data, which are multi-scale features. Multi-scale image data refers to representations of the same image at different resolutions or local regions, capturing hierarchical information from global semantics to local details. Furthermore, semantic attributes of the multi-viewpoint image data can be generated by processing the third-image feature data.

[0053] In one embodiment of the present invention, the fused image feature information and semantic attributes can be input into the target model to output three-dimensional scene data, which can specifically include: obtaining time information corresponding to multi-view image data; inputting time information, fused image feature information and semantic attributes into the target model to output three-dimensional scene data.

[0054] In practical applications, the time information corresponding to the multi-view image data can be obtained, and then the time information, fused image feature information and semantic attributes can be simultaneously input into the target model to output three-dimensional scene data.

[0055] In an embodiment of the present invention, dynamic scene processing and the integration of the time dimension can be achieved. Specifically, by adding a time variable to the NeRF model, continuous modeling of dynamic targets can be achieved, ensuring the temporal continuity and spatial consistency of the dynamic scene.

[0056] Compared with traditional technologies: Existing NeRF models are mostly limited to static scenes. This technical solution can effectively handle the motion state of dynamic elements (such as pedestrians and vehicles), significantly improving its applicability and accuracy in actual autonomous driving tests.

[0057] In one embodiment of the present invention, generating autonomous driving image data based on three-dimensional scene data includes: creating a physical model based on preset climate events, the physical model being used to simulate the interaction between light transmission and the environment to produce different weather effects; generating simulated weather effects based on the physical model; and combining the simulated weather effects with the three-dimensional scene data to generate autonomous driving image data. The climate events can be set based on actual scene requirements.

[0058] In the embodiment of the present invention, the environmental simulation can be enhanced, and the simulation of various weather conditions and lighting conditions can be supported through the physical model, and test image data under different scenarios can be generated through parameter control.

[0059] Compared with traditional technologies: Traditional image generation methods (such as diffusion models and GANs) are prone to training instability, artifacts, or a monotonous style when generating diverse scenes. This technical solution achieves both data diversity and realism by accurately simulating actual driving environments, enriching the scenarios of autonomous driving test data.

[0060] In one embodiment of the present invention, after acquiring multi-view image data collected by a target vehicle in a road scene, the method includes: Preprocess the multi-view image data, including any one or more of the following: Data labeling processing, image denoising and distortion correction, and data cleaning.

[0061] Among them, the data labeling process is to record the timestamp, sensor position information and posture data of each frame of multi-view image data; the image denoising and distortion correction process is to filter out noise and correct lens distortion of multi-view image data; the data cleaning preprocessing process is to remove fuzzy, redundant or incorrectly labeled data in the multi-view image data.

[0062] In an embodiment of the present invention, on the one hand, by introducing the semantic attributes of the image, it is possible to integrate the semantic information of key targets such as roads, vehicles, and pedestrians into the modeling process. Compared to the diffusion model, GAN model or standard NeRF that only recovers information from the image, the embodiment of the present invention can add semantic information during the neural rendering process, thereby improving the accuracy of target recognition and detail reconstruction, generating images that conform to reality, and achieving precise control of the generated results. On the other hand, a dynamic weighting mechanism can be used to assign weights based on image clarity, exposure, and overlapping area information to achieve optimal fusion of data from each perspective. Compared to traditional three-dimensional reconstruction methods that usually rely on a single or a few perspective data, or only perform simple weighted processing, which makes it difficult to fully utilize multi-angle information, the embodiment of the present invention improves modeling accuracy through dynamic weight allocation, and has higher robustness and adaptability.

[0063] Reference Figure 3 , is a schematic diagram of generating three-dimensional image data in an embodiment of the present invention, which may specifically include: a data processing stage, a scene reconstruction stage, and a data generation stage.

[0064] Among them, the data processing stage can include: (1) data collection and preprocessing; (2) multi-perspective fusion and dynamic weight calculation; (3) semantic familiarity integration; (2) dynamic scene processing and time dimension integration.

[0065] Specifically, during the data acquisition and preprocessing process, vehicle-mounted camera data can be used to collect multi-view image data, mark the data timestamp / pose, and perform denoising, distortion correction, and data cleaning on the image data.

[0066] In the process of multi-view fusion and dynamic weight calculation, feature extraction and perspective analysis of multi-view image data can be performed to calculate the overlapping area information, and then a dynamic weight mechanism can be used to dynamically allocate weights to generate fused image features.

[0067] In the process of semantic attribute integration, CLIP can be used to generate multi-scale features for image data, capture semantic content from global structure to local details, and add semantic attributes to the image.

[0068] In the process of integrating dynamic scene processing with the time dimension, time t can be added as the input of the target model, allowing the network to learn the changes of the scene over time. A two-stage network learning is set up, in which the normalized network represents the static scene and the deformed network handles the time changes.

[0069] After completing the aforementioned data processing, NeRF scene reconstruction can be performed. Specifically, through the aforementioned steps of data preparation, network definition, and volume rendering, 2D images can be converted into a continuous 3D scene representation, achieving high-quality reconstruction and rendering, and predicting the color, density, and semantic information of each point. The generated 3D image data can be optimized in a further step, namely, test data generation and enhancement. Specifically, physical models can be designed based on target climate events (such as fog and snow) to simulate light transmission and environmental interaction. This simulated weather effect can then be combined with the original NeRF scene to simulate scene effects under different weather and lighting conditions, generating diverse test data.

[0070] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0071] Reference Figure 4 , which shows a schematic structural diagram of a device for generating image data for autonomous driving provided by an embodiment of the present invention, which may specifically include the following modules: The multi-view image data acquisition module 401 is used to acquire multi-view image data collected by the target vehicle in the road scene; A feature fusion module 402 is configured to perform feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generate fused image feature information based on the plurality of first image feature information; A semantic attribute extraction module 403, configured to generate semantic attributes based on the multi-view image data; A three-dimensional scene data output module 404 is configured to input the fused image feature information and the semantic attributes into a target model and output three-dimensional scene data; The image data generation module 405 is used to generate autonomous driving image data according to the three-dimensional scene data.

[0072] In one embodiment of the present invention, the feature fusion module 402 may include the following submodules: A weight information determination submodule, configured to determine weight information corresponding to each first image feature information based on a preset dynamic weight mechanism; A second image feature data generating submodule is configured to perform weighted processing on the weight information and the corresponding first image feature information to obtain second image feature data; The image fusion submodule is used to fuse multiple second image feature data to generate fused image feature information.

[0073] In one embodiment of the present invention, the semantic attribute extraction module 403 may include the following submodules: A third image feature extraction submodule is used to input the multi-view image data into the CLIP model to extract third image feature data; The semantic attribute generation submodule is configured to generate semantic attributes of the multi-view image data based on the third image feature data.

[0074] In one embodiment of the present invention, the 3D scene data output module 404 may include the following submodules: A time information acquisition submodule, configured to acquire time information corresponding to the multi-view image data; The three-dimensional scene data output submodule is used to input the time information, the fused image feature information and the semantic attributes into the target model and output three-dimensional scene data.

[0075] In one embodiment of the present invention, the test image data generation module 405 may include the following submodules: A physical model creation submodule is used to create a physical model based on a preset climate event. The physical model is used to simulate the interaction between light transmission and the environment to obtain different weather effects. A weather effect simulation submodule, configured to generate simulated weather effects based on the physical model; The autonomous driving image data generation submodule is used to combine the simulated weather effect with the three-dimensional scene data to generate autonomous driving image data.

[0076] In one embodiment of the present invention, the apparatus may include: A preprocessing module is used to preprocess the multi-view image data, wherein the preprocessing includes any one or more of the following: Data labeling processing, image denoising and distortion correction, and data cleaning.

[0077] In one embodiment of the present invention, the first image feature information includes any one or more of the following: Image clarity information, viewing angle coverage information, and overlapping area information.

[0078] In this embodiment of the present invention, by incorporating semantic attributes into images, semantic information about key objects such as roads, vehicles, and pedestrians can be incorporated into the modeling process. Compared to diffusion models, GAN models, or standard NeRF, which only recover information from images, this embodiment of the present invention incorporates semantic information into the neural rendering process, thereby improving the accuracy of object recognition and detail reconstruction, generating realistic images, and enabling precise control over the generated results.

[0079] An embodiment of the present invention also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the above-mentioned method for generating autonomous driving image data is implemented.

[0080] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for generating autonomous driving image data as described above is implemented.

[0081] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0082] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0083] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0084] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0087] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0088] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0089] The above is a detailed introduction to the provided method, device, equipment, and medium for generating autonomous driving image data. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, based on the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A method for generating autonomous driving image data, characterized in that: The method comprises: Acquire multi-view image data collected by the target vehicle in the road scene; Performing feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generating fused image feature information based on the plurality of first image feature information; generating semantic attributes based on the multi-view image data; Inputting the fused image feature information and the semantic attributes into a target model, and outputting three-dimensional scene data; Automatic driving image data is generated according to the three-dimensional scene data.

2. The method according to claim 1, characterized in that The generating of fused image feature information based on the plurality of first image feature information includes: Determining weight information corresponding to each first image feature information based on a preset dynamic weight mechanism; Performing weighted processing on the weight information and the corresponding first image feature information to obtain second image feature data; The plurality of second image feature data are fused to generate fused image feature information.

3. The method according to claim 1, characterized in that Generating semantic attributes according to the multi-view image data includes: Inputting the multi-view image data into a CLIP model to extract third image feature data; Semantic attributes of the multi-view image data are generated based on the third image feature data.

4. The method according to claim 1, wherein The step of inputting the fused image feature information and the semantic attributes into a target model and outputting three-dimensional scene data includes: Acquiring time information corresponding to the multi-view image data; The time information, the fused image feature information, and the semantic attributes are input into a target model, and three-dimensional scene data is output.

5. The method according to claim 1, wherein Generating the autonomous driving image data according to the three-dimensional scene data includes: Creating a physical model based on preset climate events, wherein the physical model is used to simulate the interaction between light transmission and the environment to obtain different weather effects; generating simulated weather effects based on the physical model; The simulated weather effect is combined with the three-dimensional scene data to generate autonomous driving image data.

6. The method according to claim 1, characterized in that After acquiring multi-view image data collected by the target vehicle in the road scene, including: Preprocessing the multi-view image data includes any one or more of the following: Data labeling processing, image denoising and distortion correction, and data cleaning.

7. The method according to claim 1, characterized in that The first image feature information includes any one or more of the following: Image clarity information, viewing angle coverage information, and overlapping area information.

8. A device for generating image data for autonomous driving, characterized in that: The device comprises: A multi-view image data acquisition module is used to acquire multi-view image data collected by the target vehicle in the road scene; a feature fusion module, configured to perform feature extraction based on the multi-view image data to obtain a plurality of first image feature information, and generate fused image feature information based on the plurality of first image feature information; a semantic attribute extraction module, configured to generate semantic attributes based on the multi-view image data; a three-dimensional scene data output module, configured to input the fused image feature information and the semantic attributes into a target model and output three-dimensional scene data; An image data generation module is used to generate autonomous driving image data according to the three-dimensional scene data.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for generating autonomous driving image data as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for generating autonomous driving image data as described in any one of claims 1 to 7.

Images