Generative model training method and device and digital vehicle asset generation method and device
By replacing the 3D Gaussian sputtering decoder with an MLP in the Trellis model and combining it with LoRA fine-tuning, the generative model was trained, solving the problem of time-consuming and labor-intensive manual modeling. This enabled the generation of high-fidelity digital vehicle assets, which are suitable for autonomous driving simulation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-17
- Publication Date
- 2026-03-27
AI Technical Summary
In existing technologies, manually modeling and generating vehicle assets is time-consuming and labor-intensive, making it difficult to build a sufficiently large high-fidelity digital asset library in a short period of time, and it is also difficult to simulate real lighting effects.
The output layer of the 3D Gaussian sputtering decoder in the Trellis model is replaced with a multilayer perceptron (MLP), and combined with a LoRA fine-tuning structure, to train a generative model to simulate lighting effects and generate high-fidelity digital vehicle assets.
While reducing time and labor costs, it generates high-fidelity digital vehicle assets suitable for autonomous driving scenarios, capable of simulating lighting effects from different angles.
Smart Images
Figure CN121746570A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of autonomous driving simulation, in particular to a generation model training method and device, and a digital vehicle asset generation method and device. BACKGROUND
[0002] A high-fidelity vehicle asset is one of the core components of any autonomous driving high-fidelity system, and one of the important factors for an autonomous driving simulation system to achieve high fidelity is whether the vehicle asset itself is sufficiently realistic. High-fidelity of the vehicle asset requires the model to be reasonable in structure and to conform to real road conditions in terms of lighting and texture.
[0003] The prior art mainly relies on experienced modelers to manually design models by spending a lot of manpower and time cost, so that the models have sufficient fidelity in physics and lighting. Artificial processing cannot guarantee the uniform quality of the modeled vehicle assets, and it is difficult to establish a large enough digital asset library in a short period of time for the autonomous driving simulation system. SUMMARY
[0004] To solve the above technical problems, the present application provides a digital vehicle asset generation model training method and device, a digital vehicle asset generation method and device, an electronic device, a storage medium and a computer program product.
[0005] According to a first aspect of the present application, a digital vehicle asset generation model training method is provided, comprising: obtaining a pre-collected high-fidelity real 3D vehicle asset; replacing the output layer of the 3D Gaussian sputtering decoder contained in the Trellis model with an M-layer MLP (Multilayer Perceptron), to obtain a to-be-trained generation model; wherein the spherical harmonic coefficient order of the 3D Gaussian sputtering output by the to-be-trained generation model is N; M is a positive integer, and N is a positive integer; In the process of training the to-be-trained generation model using the 3D vehicle asset, the network parameters in the M-layer MLP are adjusted, and other network parameters are kept fixed, to generate a digital vehicle asset generation model.
[0006] Optionally, the method further comprises: Before obtaining the to-be-trained generation model, a LoRA (Low-Rank Adaptation) fine-tuning structure is added to the linear projection layer before each attention module in the Trellis model; In the process of training the to-be-trained generation model using the 3D vehicle asset, the network parameters in the LoRA fine-tuning structure are adjusted.
[0007] Optionally, the spherical harmonic coefficient order N of the 3D Gaussian sputtering is 3.
[0008] According to a second aspect of the present application, a digital vehicle asset generation method is provided, comprising: Data acquisition is performed on a to-be-simulated road surface to obtain an original road condition image sequence; An image sequence of a target vehicle is segmented from the original road condition image sequence; The image sequence of the target vehicle is input into a pre-trained digital vehicle asset generation model to obtain a digital vehicle asset of the target vehicle; wherein the digital vehicle asset generation model is trained based on the method of the first aspect.
[0009] Optionally, the digital vehicle asset generation method further comprises: After obtaining the digital vehicle asset of the target vehicle, quality detection is performed on the digital vehicle asset of the target vehicle to obtain a vehicle asset fidelity; When the vehicle asset fidelity is less than a preset fidelity, the digital vehicle asset of the target vehicle is deleted; When the vehicle asset fidelity is greater than or equal to the preset fidelity, the digital vehicle asset of the target vehicle is added to a vehicle asset database.
[0010] According to a third aspect of the present application, a digital vehicle asset generation model training device is provided, comprising: A real 3D vehicle asset acquisition module is configured to acquire pre-collected high-fidelity real 3D vehicle assets; A to-be-trained generation model construction module is configured to replace the output layer of the 3D Gaussian sputtering decoder contained in the Trellis model with an M-layer MLP to obtain a to-be-trained generation model; wherein the spherical harmonic coefficient order N of the 3D Gaussian sputtering output by the to-be-trained generation model is N; M is a positive integer, and N is a positive integer; A network parameter adjustment module is configured to adjust the network parameters in the M-layer MLP and keep other network parameters fixed during training of the to-be-trained generation model using the 3D vehicle assets, and generate a digital vehicle asset generation model.
[0011] Optionally, the to-be-trained generation model construction module is further configured to add a LoRA fine-tuning structure to the linear projection layer before each attention module in the Trellis model; The network parameter adjustment module is further configured to adjust the network parameters in the LoRA fine-tuning structure during training of the to-be-trained generation model using the 3D vehicle assets.
[0012] Optionally, the spherical harmonic coefficient order N of the 3D Gaussian sputtering is 3.
[0013] According to a fourth aspect of the present application, a digital vehicle asset generation apparatus is provided, comprising: a data collection module configured to collect data of a road surface to be simulated to obtain an original road condition image sequence; an image segmentation module configured to segment an image sequence of a target vehicle from the original road condition image sequence; a digital vehicle asset generation module configured to input the image sequence of the target vehicle into a pre-trained digital vehicle asset generation model to obtain a digital vehicle asset of the target vehicle; wherein the digital vehicle asset generation model is trained based on the method of the first aspect.
[0014] Optionally, the digital vehicle asset generation apparatus further comprises: a quality detection module configured to perform quality detection on the digital vehicle asset of the target vehicle after obtaining the digital vehicle asset of the target vehicle to obtain a vehicle asset fidelity; a digital vehicle asset deletion module configured to delete the digital vehicle asset of the target vehicle when the vehicle asset fidelity is less than a preset fidelity; a digital vehicle asset storage module configured to add the digital vehicle asset of the target vehicle to a vehicle asset database when the vehicle asset fidelity is greater than or equal to the preset fidelity.
[0015] According to a fifth aspect of the present application, an electronic device is provided, comprising: a processor configured to execute a computer program stored in a memory, the computer program being executed by the processor to implement the method of the first aspect or the second aspect.
[0016] According to a sixth aspect of the present application, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the method of the first aspect or the second aspect.
[0017] According to a seventh aspect of the present application, a computer program product is provided, which, when running on a computer, causes the computer to execute the method of the first aspect or the second aspect.
[0018] The technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art: The 3D Gaussian sputtering decoder part of the Trellis model is redesigned, i.e., the output layer of the 3D Gaussian sputtering decoder is replaced by an M-layer MLP, so that the order of the final 3D Gaussian sputtering spherical harmonic coefficient is increased from 0 to N (for example, 3). Based on the model architecture, a digital vehicle asset generation model is trained, which can not only restore the color, but also simulate the lighting effect at different angles, and can be applied to real automatic driving scenes. Using the digital vehicle asset generation model to automatically generate digital vehicle assets can greatly generate high-fidelity digital assets required by automatic driving simulation scenes while reducing a large amount of time and labor costs. BRIEF DESCRIPTION OF DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application, together with the description.
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.
[0021] Figure 1 A structural schematic diagram of a digital vehicle asset generation model in an embodiment of the present application; Figure 2 A flowchart of a digital vehicle asset generation model training method in an embodiment of the present application; Referring to Figure 3 , Figure 3 Another flowchart of a digital vehicle asset generation method in an embodiment of the present application; Figure 4 A partial structural schematic diagram of a digital vehicle asset generation model in an embodiment of the present application; Figure 5 A flowchart of a digital vehicle asset generation method in an embodiment of the present application; Figure 6 A flowchart of a digital vehicle asset generation model training device in an embodiment of the present application; Figure 7 A structural schematic diagram of a digital vehicle asset generation device in an embodiment of the present application; Figure 8 A structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0022] In order to enable the above-mentioned purposes, features and advantages of the present application to be more clearly understood, the solutions of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0023] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in other manners different from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present application, not all the embodiments.
[0024] In order to be able to generate high-fidelity vehicle digital assets from collected road condition data at low cost and in large quantities for subsequent simulation systems. The embodiments of the present application utilize an open-source pre-trained large model, i.e., the Trellis model, and combine high-fidelity modeling of real 3D vehicle assets to train a digital vehicle asset generation model that is specifically used to generate high-fidelity digital vehicle assets.
[0025] For the open-source Trellis model, the standard representation format of its digital assets is 3DGS (3D Gaussian Splatting) parameters with a spherical harmonic coefficient order of 0, and the parameter amount of each 3D Gaussian sphere is 14 dimensions. The standard 3DGS with a spherical harmonic coefficient order of 0 can well represent the color and texture of the 3D model in animation, artistic, and other scenes. Since the 3DGS with a spherical harmonic coefficient order of 0 can only restore color and cannot simulate the lighting effect at different angles, it is helpless for the lighting of real autonomous driving scenes.
[0026] In order to generate high-fidelity digital vehicle assets required by autonomous driving simulation scenes to the greatest extent, the embodiments of the present application redesign the 3DGS decoder part of the Trellis model on the basis of the Trellis model. Figure 1 The structure diagram of the digital vehicle asset generation model of the embodiments of the present application is shown as Figure 1 As shown, the output layer of the original 3D Gaussian Splatting decoder is replaced by a multi-layer MLP, which increases the output dimension and adds additional dimensions to the high-order spherical harmonic coefficient. In this way, the spherical harmonic coefficient order of the 3D Gaussian Splatting is increased to N, thereby better simulating the lighting effect of the vehicle.
[0027] Referring to Figure 2 , Figure 2 A flowchart of the digital vehicle asset generation method in the embodiments of the present application can include the following steps: Step S202, acquiring pre-collected high-fidelity real 3D vehicle assets.
[0028] In order to improve the fidelity of the finally generated digital vehicle asset generation model, a sufficiently realistic digital vehicle asset can be constructed. Specifically, high-fidelity real 3D vehicle assets can be collected on model communities such as Free3D, Sketchfab, and CGTrader.
[0029] In step S204, the output layer of the 3D Gaussian Splat decoder contained in the Trellis model is replaced with an M-layer MLP to obtain a to-be-trained generation model; wherein the order of the spherical harmonic coefficients of the 3D Gaussian Splat output by the to-be-trained generation model is N.
[0030] The structure of the to-be-trained generation model can be referred to as Figure 1 The number of layers M of the MLP is a positive integer. The larger M is, the more network parameters need to be adjusted during model training, and the larger the calculation amount is. The smaller M is, the lower the fidelity of the finally generated digital vehicle asset generation model is. Therefore, a suitable value can be selected by comprehensively considering the calculation amount and the fidelity of the model, for example, M can be 3, etc. Figure 1 The model structure shown in the figure is a three-layer MLP. N is also a positive integer. Optionally, the order N of the spherical harmonic coefficients of the 3D Gaussian Splat is 3, which can make the simulated lighting effect optimal.
[0031] In step S206, during the training of the to-be-trained generation model using the 3D vehicle asset, the network parameters in the M-layer MLP are adjusted, and other network parameters are kept fixed to generate a digital vehicle asset generation model.
[0032] The model training of the embodiments of the present application is generative training. By making the model learn the distribution of the data, a new and realistic data sample similar to the training data can be generated. During the training process, only the network parameters in the M-layer MLP are adjusted, and other network parameters remain unchanged, so that the finally generated digital vehicle asset generation model can be applicable to the autonomous driving scene.
[0033] Referring to Figure 3 , Figure 3 is another flowchart of the digital vehicle asset generation method in the embodiments of the present application, comprising the following steps: In step S302, high-fidelity real 3D vehicle assets collected in advance are obtained.
[0034] In step S304, the output layer of the 3D Gaussian Splat decoder contained in the Trellis model is replaced with an M-layer MLP, and a LoRA fine-tuning structure is added to the linear projection layer before each attention module in the Trellis model to obtain a to-be-trained generation model; wherein the order of the spherical harmonic coefficients of the 3D Gaussian Splat output by the to-be-trained generation model is N.
[0035] In addition to replacing the output layer of the 3D Gaussian sputtering decoder, the LoRA method can be used to fine-tune the model, and a LoRA fine-tuning structure is added before each attention module. For example, the LoRA method can be used to fine-tune the model sparse structure generation module (Sparse Flow Transformer) and the sparse structure autoencoder (Sparse VAE Encoder).
[0036] As shown in Figure 4 , for a single attention module, a LoRA fine-tuning structure is added to the linear projection layer before the attention module, X is the input, W is the pre-training weight of the linear projection layer, d is the dimension of the pre-training weight W, r is the rank of the selected low-rank matrix, q represents the query vector, k represents the key vector, and v represents the numerical vector. The training data of the Trellis model contains a part of the vehicle data, and the Trellis model can inherit the ability. After adding the LoRA fine-tuning structure, a digital vehicle asset generation model with better effect can be trained.
[0037] Step S306, in the process of training the to-be-trained generation model using the 3D vehicle asset, the network parameters in the M-layer MLP and the network parameters in the LoRA fine-tuning structure are adjusted, and other network parameters are fixed, to generate a digital vehicle asset generation model.
[0038] Similarly, during the training process, the inference path of the Trellis model is kept unchanged, only the network parameters in the M-layer MLP and the LoRA fine-tuning structure, and the weighting coefficients of each LoRA fine-tuning structure and the linear projection layer are adjusted, and other network parameters are kept unchanged, so that the finally generated digital vehicle asset generation model is more suitable for the automatic driving scene.
[0039] The digital vehicle asset generation method of the embodiment of the present application trains a digital vehicle asset generation model based on the above-mentioned model architecture, which not only restores the color but also simulates the lighting effect at different angles, and can be applied to real automatic driving scenes. Using the digital vehicle asset generation model to automatically generate digital vehicle assets can greatly reduce the time and labor costs while generating high-fidelity digital assets required for automatic driving simulation scenes.
[0040] The embodiment of the present application also provides a digital vehicle asset generation method, as shown in Figure 5 , which can include the following steps: Step S502, data acquisition is performed on the to-be-simulated road surface to obtain an original road image sequence.
[0041] Step S504, the image sequence of the target vehicle is segmented from the original road image sequence.
[0042] The target vehicle refers to a vehicle to be generated with a digital vehicle asset, and an AI (Artificial Intelligence) segmentation large model can be used to segment an image sequence of the target vehicle from an original road image sequence.
[0043] In step S506, the image sequence of the target vehicle is input into a pre-trained digital vehicle asset generation model to obtain a digital vehicle asset of the target vehicle; wherein the digital vehicle asset generation model is based on Figure 2 or Figure 3 The method is trained.
[0044] In the embodiment of the application, the pre-trained digital vehicle asset generation model is used to obtain the digital vehicle asset of the target vehicle in the form of a graph generation model.
[0045] Optionally, after obtaining the digital vehicle asset of the target vehicle, quality detection is performed on the digital vehicle asset of the target vehicle to obtain a vehicle asset fidelity; when the vehicle asset fidelity is less than a preset fidelity, it indicates that the fidelity of the digital vehicle asset of the target vehicle is too low, and the digital vehicle asset of the target vehicle is deleted; when the vehicle asset fidelity is greater than or equal to the preset fidelity, the digital vehicle asset of the target vehicle is added to the vehicle asset database. Alternatively, limited correction and adjustment can also be performed manually to obtain a digital vehicle asset with higher quality, which is stored in the vehicle asset database.
[0046] The digital vehicle asset generation method of the embodiment of the application relies on an AI large model, and can generate high-quality digital vehicle assets with high consistency while reducing a large amount of time and labor costs.
[0047] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.
[0048] The embodiment of the application also provides a digital vehicle asset generation model training device, which is described with reference to Figure 6 The digital vehicle asset generation model training device 600 comprises: A real 3D vehicle asset acquisition module 602 is configured to acquire pre-collected high-fidelity real 3D vehicle assets. The to-be-trained generation model construction module 604 is configured to replace an output layer of a 3D Gaussian sputtering decoder contained in the Trellis model with an M-layer MLP to obtain a to-be-trained generation model; wherein an order of spherical harmonic coefficients of the 3D Gaussian sputtering output by the to-be-trained generation model is N; M is a positive integer, and N is a positive integer; The network parameter adjustment module 606 is configured to adjust network parameters in the M-layer MLP while keeping other network parameters fixed in a process of training the to-be-trained generation model by using the 3D vehicle asset, and generate a digital vehicle asset generation model.
[0049] Optionally, the to-be-trained generation model construction module 604 is further configured to add an LoRA fine-tuning structure before a linear projection layer of each attention module in the Trellis model. The network parameter adjustment module 606 is further configured to adjust network parameters in the LoRA fine-tuning structure in the process of training the to-be-trained generation model by using the 3D vehicle asset.
[0050] Optionally, the order N of the spherical harmonic coefficients of the 3D Gaussian sputtering is 3.
[0051] Embodiments of the present application also provide a digital vehicle asset generation device, referring to Figure 7 The digital vehicle asset generation device 700 comprises: The data acquisition module 702 is configured to acquire data of a to-be-simulated road surface to obtain an original road condition image sequence. The image segmentation module 704 is configured to segment an image sequence of a target vehicle from the original road condition image sequence. The digital vehicle asset generation module 706 is configured to input the image sequence of the target vehicle into a pre-trained digital vehicle asset generation model to obtain a digital vehicle asset of the target vehicle; wherein the digital vehicle asset generation model is trained based on the method. Figure 2 Or Figure 3 .
[0052] Optionally, the digital vehicle asset generation device 700 further comprises: The quality detection module is configured to perform quality detection on the digital vehicle asset of the target vehicle after obtaining the digital vehicle asset of the target vehicle to obtain a vehicle asset fidelity. The digital vehicle asset deletion module is configured to delete the digital vehicle asset of the target vehicle when the vehicle asset fidelity is less than a preset fidelity. The digital vehicle asset storage module is configured to add the digital vehicle asset of the target vehicle to a vehicle asset database when the vehicle asset fidelity is greater than or equal to the preset fidelity.
[0053] The specific details of each module or unit in the above apparatus have been described in detail in the corresponding method, and therefore will not be described here.
[0054] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into embodied by multiple modules or units.
[0055] The embodiments of the present application also provide an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the above-mentioned digital vehicle asset generation model training method or digital vehicle asset generation method.
[0056] Referring to Figure 8 , Figure 8 A structure schematic diagram of an electronic device in the embodiments of the present application, the embodiments of the present application do not limit the specific implementation of the electronic device.
[0057] As Figure 8 indicated, the electronic device can include: a processor 802, a communication interface 804, a memory 806, and a communication bus 808.
[0058] The processor 802, the communication interface 804, and the memory 806 complete mutual communication through the communication bus 808.
[0059] The communication interface 804 is configured to communicate with other electronic devices or servers.
[0060] The processor 802 is configured to execute the program 810, and specifically can execute the related steps in the above method embodiments.
[0061] Specifically, the program 810 can include program code, which includes computer operation instructions.
[0062] The processor 802 can be a central processing unit, or a specific integrated circuit, or one or more integrated circuits configured to implement one or more embodiments of the present application. One or more processors included in the smart device can be the same type of processor, such as one or more CPUs; or can be different types of processors, such as one or more CPUs and one or more ASICs.
[0063] A memory 806 for storing a program 810. The memory 806 can include a high-speed RAM memory and can also include a non-volatile memory, such as at least one disk memory.
[0064] The program 810 can specifically be used to cause the processor 802 to perform the steps in the above-described digital vehicle asset generation model training method or digital vehicle asset generation method embodiments.
[0065] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be described here.
[0066] In the embodiments of the present application, a computer readable storage medium having a computer program stored thereon is also provided, and the computer program is executed by a processor to implement the above-described digital vehicle asset generation model training method or digital vehicle asset generation method.
[0067] It should be noted that the computer readable storage medium shown in the present application can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. The program code contained in the computer readable storage medium can be transmitted in any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency, etc., or any suitable combination of the above.
[0068] In the embodiments of the present application, a computer program product is also provided, which, when running on a computer, causes the computer to execute the above-described digital vehicle asset generation model training method or digital vehicle asset generation method.
[0069] It has to be noted that, in the present document, relational terms are intended only to convey a possible relationship between elements or
[0070] The foregoing detailed description has set forth various embodiments of the application via the use of specific terminology. However, embodiments thereof can be practiced without the specific details ("every" embodiment) set forth above. In general, the teachings of the present application can be applied to any suitable industrial setting and / or process of this type. The foregoing description, therefore, is not intended to be limiting, but merely illustrative. Further, the herein disclosed subject matter is intended to cover all alternatives, modifications and equivalents. Thus, the scope of the present application should be determined by the appended claims and their legal equivalents, and not by the ability or inability to reduce the present application to practice according to the provisions of the patent statute.
Claims
1. A training method for a digital vehicle asset generation model, characterized in that, include: Acquire pre-collected high-fidelity real 3D vehicle assets; The output layer of the 3D Gaussian sputtering decoder in the Trellis model is replaced with M multilayer perceptrons (MLPs) to obtain the generative model to be trained; wherein, the order of the spherical harmonic coefficients of the 3D Gaussian sputtering output by the generative model to be trained is N; M is a positive integer and N is a positive integer; During the training of the model to be trained using the 3D vehicle assets, the network parameters in the M-layer MLP are adjusted while keeping other network parameters fixed to generate a digital vehicle asset generation model.
2. The method according to claim 1, characterized in that, The method further includes: Before obtaining the generative model to be trained, a low-rank adaptive LoRA fine-tuning structure is added to the linear projection layer before each attention module in the Trellis model. During the training of the model to be trained using the 3D vehicle assets, the network parameters in the LoRA fine-tuning structure are adjusted.
3. The method according to claim 1 or 2, characterized in that, The spherical harmonic order N of the 3D Gaussian sputtering is 3.
4. A method for generating digital vehicle assets, characterized in that, include: Data is collected from the simulated road surface to obtain the original road condition image sequence; Segment the target vehicle image sequence from the original road condition image sequence; The image sequence of the target vehicle is input into a pre-trained digital vehicle asset generation model to obtain the digital vehicle asset of the target vehicle; wherein the digital vehicle asset generation model is trained based on the method described in claims 1 to 3.
5. The method according to claim 4, characterized in that, The method further includes: After obtaining the digital vehicle assets of the target vehicle, the digital vehicle assets of the target vehicle are subjected to quality testing to obtain the vehicle asset fidelity. When the fidelity of the vehicle asset is less than the preset fidelity, delete the digital vehicle asset of the target vehicle; When the fidelity of the vehicle asset is greater than or equal to the preset fidelity, the digital vehicle asset of the target vehicle is added to the vehicle asset database.
6. A training device for a digital vehicle asset generation model, characterized in that, include: The Real 3D Vehicle Asset Acquisition Module is used to acquire pre-collected high-fidelity real 3D vehicle assets. The module for building a model to be trained is used to replace the output layer of the 3D Gaussian sputtering decoder in the Trellis model with M multilayer perceptrons (MLPs) to obtain a model to be trained; wherein, the order of the spherical harmonic coefficients of the 3D Gaussian sputtering output by the model to be trained is N; M is a positive integer and N is a positive integer; The network parameter adjustment module is used to adjust the network parameters in the M-layer MLP during the training of the generator model using the 3D vehicle assets, while keeping other network parameters fixed, to generate a digital vehicle asset generation model.
7. The apparatus according to claim 6, characterized in that, The module for building the model to be trained is also used to add a low-rank adaptive LoRA fine-tuning structure to the linear projection layer before each attention module in the Trellis model. The network parameter adjustment module is also used to adjust the network parameters in the LoRA fine-tuning structure during the training of the model to be trained using the 3D vehicle assets.
8. The apparatus according to claim 6 or 7, characterized in that, The spherical harmonic order N of the 3D Gaussian sputtering is 3.
9. A digital vehicle asset generation device, characterized in that, include: The data acquisition module is used to collect data on the simulated road surface and obtain the original road condition image sequence. An image segmentation module is used to segment an image sequence of the target vehicle from the original road condition image sequence; A digital vehicle asset generation module is used to input an image sequence of a target vehicle into a pre-trained digital vehicle asset generation model to obtain digital vehicle assets of the target vehicle; wherein, the digital vehicle asset generation model is trained based on the method described in claims 1 to 3.
10. The apparatus according to claim 9, characterized in that, The device further includes: The quality inspection module is used to perform quality inspection on the digital vehicle assets of the target vehicle after obtaining the digital vehicle assets of the target vehicle, and to obtain the fidelity of the vehicle assets. The digital vehicle asset deletion module is used to delete the digital vehicle assets of the target vehicle when the fidelity of the vehicle asset is less than a preset fidelity. The digital vehicle asset storage module is used to add the digital vehicle assets of the target vehicle to the vehicle asset database when the fidelity of the vehicle asset is greater than or equal to a preset fidelity.