Automobile three-view generation method, device and equipment based on multi-view splicing training

Abstract

The invention discloses an automobile three-view generation method, device and equipment based on multi-view splicing training, and relates to the technical field of computer vision and image generation, and the method comprises the steps: collecting three-view pictures of an automobile to construct an image set, and carrying out the labeling of the image set, and obtaining a training set; and training a LoRA model based on the training set, and adjusting a FLUX model based on the trained LoRA model, thereby realizing three-view generation of the target vehicle based on the adjusted FLUX model. According to the method and the device, the three views of the same vehicle can be generated at one time when the automobile design drawing is drawn, the consistency among the three views is ensured, the generated image is high in quality and rich in details, and an efficient and convenient vehicle three-view generation scheme is provided for a user.

Description

Technical Field

[0001] The present application relates to the field of computer vision and image generation technology, and specifically to a method, device and equipment for generating three-view images of a car based on multi-view stitching training. Background Art

[0002] In the field of automotive design, exterior designers are often required to draw renderings from three perspectives suitable for automotive styling reviews: the front left or right 45° perspective, the front side perspective, and the rear left or right 45° perspective to showcase the vehicle's exterior design. Traditional three-view drawing methods rely on manual drawing by designers, which is time-consuming and labor-intensive. With the development of artificial intelligence technology, image generation technology based on generative models has gradually been applied to the automotive design field.

[0003] Traditional AI-assisted content generation technology in the automotive design process can only produce images from a single perspective, resulting in discrete generation of views. Images from different perspectives may differ in shape, proportion, detail, etc. These differences may cause unnecessary trouble and cost to subsequent design, manufacturing, and production. The single-image generation technology route also causes images used as training set materials to be stored independently, failing to establish mapping between views, ultimately leading to low production efficiency.

[0004] At the same time, the existing task of generating multiple perspectives for a unified subject often requires the use of a three-dimensional data technology route. The image quality cannot meet the designer's rendering requirements. When the generative model handles complex three-view generation tasks, there are still problems such as inconsistent generated image quality and loss of details. Summary of the Invention

[0005] The present application provides a method, device and equipment for generating three views of a car based on multi-view stitching training, which enables the generation of three views of the same vehicle at one time when drawing a car design drawing, and ensures the consistency between the three views. The generated images are of high quality and rich in details, providing users with an efficient and convenient solution for generating three views of the car.

[0006] In a first aspect, an embodiment of the present application provides a method for generating three views of a car based on multi-view stitching training, the method comprising:

[0007] Collecting three-view images of the vehicle to construct an image set, and labeling the image set to obtain a training set;

[0008] The LoRA model is trained based on the training set, and the FLUX model is adjusted based on the trained LoRA model, so as to realize three-view generation of the target vehicle based on the adjusted FLUX model.

[0009] In conjunction with the first aspect, in one embodiment, collecting three-view images of the vehicle to construct an image set, and labeling the image set to obtain a training set, specifically includes:

[0010] Acquire vehicle images, where the acquired vehicle images include multiple vehicle models, and each vehicle model includes three views;

[0011] With the vehicle body as the center, the three-view images of the same vehicle model are cropped, and each image is cropped to a preset ratio;

[0012] The cropped three-view images of the same vehicle model are stitched together in sequence to form a vertically long image, thereby obtaining vertically long images of all vehicle models and constructing an image set.

[0013] In conjunction with the first aspect, in one embodiment, labeling the image set to obtain a training set specifically includes:

[0014] The trigger words are set as text prompts, and the pictures in the image set are labeled to obtain the training set.

[0015] In combination with the first aspect, in one embodiment, training the LoRA model based on the training set specifically includes:

[0016] Use the FLUX model as the basic model for model training;

[0017] Build the LoRA model, set up the LoRA model according to the cross attention layer of the FLUX model as needed, and complete the initialization loading of the LoRA model;

[0018] The hyperparameters are set, and the LoRA model is trained using the training set to obtain a trained LoRA model.

[0019] In conjunction with the first aspect, in one embodiment, the training parameters of the LoRA model specifically include:

[0020] The rank dimension is 64, the scaling factor is 64, the matrix rank random dropout rate is 0.18, the learning rate decay scheduler type is cosine_with_restarts, the basic learning rate is 1e-4, the optimizer type is AdamW8bit, the number of gradient accumulation steps to achieve the maximum equivalent batch is 2, the number of warm-up steps is 440, the maximum training cycle is 16, and the number of samples for a single calculation is 1.

[0021] In combination with the first aspect, in one embodiment, adjusting the FLUX model based on the trained LoRA model, thereby achieving three-view generation of the target vehicle based on the adjusted FLUX model, specifically includes:

[0022] Setting text prompt words, wherein the keywords in the text prompt words include vehicle type information and environmental information;

[0023] Setting parameters, including sampling method, number of iteration steps, width and height of latent space, and prompt word guidance coefficient;

[0024] The FLUX model is adjusted based on the trained LoRA model, so that the FLUX model integrated with the LoRA model executes the generation process according to the text prompt words to realize the three-view generation of the target vehicle.

[0025] In conjunction with the first aspect, in one embodiment, adjusting the FLUX model based on the trained LoRA model specifically includes:

[0026] The latent space features are iteratively corrected through the trained LoRA model, and the geometric continuity of the three views during stitching is ensured through implicit projection transformation. The geometric shapes and positional relationships under different perspectives or in different states remain continuous and consistent, and the semantic consistency is maintained.

[0027] In conjunction with the first aspect, in one embodiment, the method for generating three-view images of a car further includes:

[0028] Three-view images of a specific type of vehicle are collected to construct an image set for labeling to obtain a specific training set. The specific training set is used to train the LoRA model to obtain a trained specific LoRA model. The FLUX model is adjusted based on the specific LoRA model, thereby realizing three-view generation of the specific type of vehicle based on the adjusted FLUX model.

[0029] In a second aspect, an embodiment of the present application provides a device for generating three views of a car based on multi-view stitching training, the device for generating three views of a car based on multi-view stitching training comprising:

[0030] A construction module is used to collect three-view images of the vehicle to construct an image set, and label the image set to obtain a training set;

[0031] An execution module is used to train the LoRA model based on the training set, and adjust the FLUX model based on the trained LoRA model, so as to realize three-view generation of the target vehicle based on the adjusted FLUX model.

[0032] In a third aspect, an embodiment of the present application provides a device for generating three views of a car based on multi-perspective stitching training, wherein the device for generating three views of a car based on multi-perspective stitching training includes a processor, a memory, and a program for generating three views of a car based on multi-perspective stitching training stored on the memory and executable by the processor, wherein when the program for generating three views of a car based on multi-perspective stitching training is executed by the processor, the steps of the method for generating three views of a car based on multi-perspective stitching training described above are implemented.

[0033] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0034] By collecting three-view images of the vehicle to build a training set and realize the training of the LoRA model, and then using the FLUX model combined with the trained LoRA model, the three-view images of the same car can be generated. This makes it possible to generate three views of the same vehicle at one time when drawing a car design drawing, and ensure the consistency between the three views. The generated images are high-quality and rich in details, providing users with an efficient and convenient vehicle three-view generation solution, and also bringing an innovative design model to the automotive design industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of a method for generating three-view images of a car based on multi-view stitching training in this application;

[0036] Figure 2 This is a schematic diagram of the functional modules of the device for generating three-view images of a car based on multi-view stitching training in this application;

[0037] Figure 3 This is a schematic diagram of the hardware structure of the automobile three-view image generation device based on multi-view stitching training in this application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0040] On the first aspect, the embodiment of the present application provides a method for generating three-view images of a car based on multi-perspective stitching training. By constructing a three-dimensional collaborative generation framework of "data-model-constraint", adopting a stitched three-view training set, and combining it with an improved LoRA training strategy, it can directly obtain three-view images of a car with good quality and corresponding features.

[0041] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of the method for generating three-view images of a car based on multi-view stitching training in this application. Figure 1 As shown in FIG, the method for generating three-view images of a car based on multi-view stitching training includes:

[0042] S1: Collect three-view images of the vehicle to construct an image set, and label the image set to obtain a training set;

[0043] S2: Train a LoRA model based on the training set, and adjust the FLUX model based on the trained LoRA model, thereby generating three views of the target vehicle based on the adjusted FLUX model. This allows for the simultaneous generation of three views of the same vehicle, i.e., three views.

[0044] Furthermore, in one embodiment, three-view images of a vehicle are collected to construct an image set, and the image set is labeled to obtain a training set, specifically including:

[0045] S101: Acquire vehicle images, where the acquired vehicle images include multiple vehicle models, and each vehicle model includes three views;

[0046] S102: Crop the three-view images of the same vehicle model with the vehicle body as the center, and crop each image to a preset ratio;

[0047] S103: The cropped three-view images of the same vehicle model are sequentially stitched together to form a vertically long image, thereby obtaining vertically long images of all vehicle models and constructing an image set.

[0048] Specifically, by collecting vehicle photographs, three perspectives of the same vehicle in the same scene can be collected and screened according to three perspectives: a 45° perspective from the left or right side of the front, a front side perspective, and a 45° perspective from the left or right side of the rear. Then, for the three views corresponding to the same vehicle, scale and crop them with the vehicle body as the center. Each image is cropped to a 2:1 ratio. Using image processing software such as Photoshop, a blank canvas with a vertical 2:3 ratio is created and divided horizontally into three grids: top, middle, and bottom. The three cropped views of the same vehicle are imported into this blank canvas, with the side view image in the middle grid and the front and rear right view images in the top and bottom grids, respectively. This results in a vertically long image corresponding to the three views of the current vehicle, thereby obtaining a vertically long image of each vehicle of each model. In other words, the two images containing the front and rear of the vehicle are separated by the front and side views, making it easier for the subsequent model to distinguish the front and rear of the vehicle.

[0049] Then, a super-resolution algorithm (such as R-ESRGAN_4x+) is used to enlarge the vertical long image to obtain clearer image quality and uniform image detail. It is foreseeable that the higher the image quality, the more conducive it is to later model training. The background of the vertical long image with a more complex background after stitching is subtracted to obtain image data with a clear subject and a clean background. All the vertical 2:3 ratio images containing the same vehicle from three perspectives (45° left or right front view, front side view, and 45° left or right rear view) after stitching, enlarging, and subtracting the background constitute the image set.

[0050] Furthermore, in one embodiment, the image set is labeled to obtain a training set, specifically including: using set trigger words as text prompts, and labeling the pictures in the image set to obtain a training set.

[0051] Specifically, describe the images in the image set, or label them. This is done by writing a text description to a .txt file corresponding to each image's file name. Each image's label simply contains the text "a trio of one car indifferent views." Repeat this process for all images in the image set to form the training set.

[0052] Furthermore, in one embodiment, training the LoRA model based on the training set specifically includes:

[0053] S201: Use the FLUX model as the basic model for model training;

[0054] S202: Build the LoRA model, set the LoRA model according to the cross attention layer of the FLUX model as needed, and complete the initialization loading of the LoRA model;

[0055] S203: Setting hyperparameters, and using the training set to train the LoRA model to obtain a trained LoRA model.

[0056] The FLUX series of models are the most advanced diffusion models, employing a more stable, controllable, and efficient approach to generate high-quality images. These improvements significantly improve image quality, speed, and cost, enabling them to be implemented directly on consumer-grade graphics cards. They can generate controllable images at 1024x1024 pixel resolution, significantly improving efficiency and quality compared to previous Stable Diffusion models.

[0057] In practical applications, this application uses the LoRA (Low-Rank Adaptation) model to fine-tune the FLUX 1.0[dev] large model.

[0058] Specifically, this application uses the open source large-scale generative pre-trained model FLUX 1.0[dev] as the base model, and adopts LoRA technology to fine-tune the model based on a prepared training set. LoRA introduces a small number of trainable parameters to adapt the model to a specific training set, reducing computational overhead and obtaining multiple LoRA-specific three-view generation models required for different vehicle project needs, such as a specific three-view model for hardcore off-road vehicles, a specific three-view model for compact cars, and a universal three-view model for multiple vehicle models. Each LoRA model can be plugged into the FLUX model to make the generated image more accurate.

[0059] It should be noted that the specific steps of using the training set to train the LoRA model include:

[0060] S21 environment configuration:

[0061] Install necessary Python libraries, such as PyTorch, diffusers, etc.

[0062] Download the open source FLUX 1.0[dev] open source model file;

[0063] Download the open source koyha model trainer;

[0064] Prepare appropriate computing resources (GPU);

[0065] S22 loads the basic model:

[0066] In this application, the LoRA model is applied to the FLUX 1.0[dev] model, so the basic model used for training is also the FLUX 1.0[dev] model;

[0067] Loading pre-trained models: FLUX 1.0[dev]

[0068] Set the output path: . / output / LoRA_models;

[0069] S23 builds the LoRA model:

[0070] Initialize the LoRA weight matrix, which contains two small weight matrices used to adjust the original weights, used to multiply the input and output respectively. Their last two dimensions are the same as the last two dimensions of the original weight tensor, and the previous dimensions are set to a smaller rank (matrix order) value (such as 4 or 8). This application chooses 4 as the rank value;

[0071] S24 training LoRA model:

[0072] Set training hyperparameters, such as batch size, learning rate, number of training rounds, etc. After experimental verification, the important parameters with better results are as follows: 1. Network structure parameter group: rank dimension (network_dim) is 64, scaling factor (network_alpha) is 64, matrix rank random dropout rate (rank_dropout) is 0.18; 2. Learning rate and optimization control parameter group: learning rate decay scheduler type (lr_scheduler) is cosine_with_restarts, basic learning rate (learning_rate) is 1e-4, optimizer type (op timizer_type) is AdamW8bit, the number of gradient accumulation steps to achieve a large equivalent batch (gradient_accumulation_steps) is 2, and the number of warm-up steps (lr_warmup_steps) is 440; 3. Training scheduling parameter group: the maximum training cycle train_epochs is 16, the number of samples for a single calculation: batch_size is 1, and an appropriate optimizer (such as AdamW) and loss function (such as CLIP loss) are used. Each time the model is run, the LoRA model weights will be updated according to the output results to make the input more accurate;

[0073] S25 Assessment and Adjustment:

[0074] Evaluate the quality of generated images;

[0075] If necessary, adjust hyperparameters or training sets and repeat training;

[0076] S26 model deployment:

[0077] Save the trained LoRA model weights.

[0078] In the actual application stage, the LoRA model is loaded into the FLUX model to generate three unified views of the target vehicle from three perspectives.

[0079] That is, in this application, the training parameters of the LoRA model specifically include:

[0080] The rank dimension is 64, the scaling factor is 64, the matrix rank random dropout rate is 0.18, the learning rate decay scheduler type is cosine_with_restarts, the basic learning rate is 1e-4, the optimizer type is AdamW8bit, the number of gradient accumulation steps to achieve the maximum equivalent batch is 2, the number of warm-up steps is 440, the maximum training cycle is 16, and the number of samples for a single calculation is 1.

[0081] Furthermore, in one embodiment, the FLUX model is adjusted based on the trained LoRA model, thereby generating three views of the target vehicle based on the adjusted FLUX model, specifically including:

[0082] S211: Setting text prompt words, wherein the keywords in the text prompt words include vehicle model information and environmental information;

[0083] S212: Setting parameters, including sampling method, number of iteration steps, width and height of latent space, and prompt word guidance coefficient;

[0084] S213: Adjust the FLUX model based on the trained LoRA model, so that the FLUX model integrated with the LoRA model performs a generation process according to the text prompt word to achieve three-view generation of the target vehicle.

[0085] Specifically, the FLUX model generation style is controlled through the trained LoRA model. Input and fine-tuning are performed based on text prompts and the LoRA model. Appropriate hyperparameters (such as step size, variance, etc.) are used to control the generation quality and feature strength. Specifically,

[0086] A1: Prepare text prompts

[0087] Set a text description to control the content and style of the generated image. Similarly, the keywords in the text prompt words include the trigger words of the LoRA model (a trio of one car in different views), vehicle model information (including descriptions of models such as sedans and SUVs), styling features (including descriptions of styling features such as "no air intake grille"), and its generation information, such as paint color and ambient lighting;

[0088] A2: Select the trained LoRA model and other styles of LoRA models

[0089] The key to generating three views of the same vehicle at once is to add training to the generation process to fine-tune the LoRA model: enter " <lora>A text description in the format of "XXXXX:0.8" can trigger the LoRA model. Additionally, you can use other styles of LoRA models to generate more creative styles based on the three perspectives while keeping the weight of the trained LoRA model set to 0.7-1.0.

[0090] A3: Set generation hyperparameters:

[0091] Set other hyperparameters as needed, such as sampling method, number of iterations, latent space width and height, and cue word guidance coefficient. The key parameter is that the ratio of latent space width to height needs to maintain the same 2:3 size ratio as the training set of the trained LoRA model. A good hyperparameter setting is as follows: sampling method (sampler): Eular; number of iterations (steps): 30; cue word guidance coefficient (CFG Scale): 3.5;

[0092] A4: Execute the build process:

[0093] Call the conditional generation function of the FLUX model;

[0094] Input includes text prompt, LoRA information, hyperparameters, etc.

[0095] The model will generate images within the specified area based on the conditions;

[0096] A5: View and evaluate build results

[0097] Evaluate whether the generated three-view image meets expectations, and adjust parameters or LoRA model weights if necessary and regenerate. It's important to note that the trigger words and size ratios corresponding to the trained LoRA model are key to triggering the FLUX model to produce images in the three-view format. The choice of hyperparameters also affects the quality and feature strength of the final result. Fine-tune and optimize the above steps based on actual needs to achieve satisfactory local results.

[0098] Furthermore, in one embodiment, the FLUX model is adjusted based on the trained LoRA model, specifically including: iteratively correcting the latent space features through the trained LoRA model, ensuring the geometric continuity when splicing the three views through implicit projection transformation, keeping the geometric shapes and positional relationships in different perspectives or different states continuous and consistent, and maintaining semantic consistency.

[0099] Specifically, in the three-view generation stage, the trained LoRA model iteratively corrects the latent space features and ensures the geometric continuity of the three views when splicing them through implicit projection transformation. For example, the geometric shapes and positional relationships of styling elements such as surfaces, wheels, and lights remain continuous and consistent at different perspectives or in different states, and maintain semantic consistency, that is, the styling styles of the front and rear of the vehicle are consistent and related.

[0100] Furthermore, in one embodiment, the automobile three-view generation method of the present application also includes: collecting three-view pictures of a specific type of vehicle to construct an image set for labeling processing to obtain a specific training set, using the specific training set to train the LoRA model to obtain a trained specific LoRA model, and adjusting the FLUX model based on the specific LoRA model, thereby realizing the three-view generation of the specific type of vehicle based on the adjusted FLUX model.

[0101] According to the above training set construction method, a training set can be constructed for a specific type of vehicle (such as sedans, SUVs, sports cars, etc.), and the LoRA model can be trained to obtain a trained specific LoRA model. Then, the FLUX model can be adjusted based on the specific LoRA model to achieve three-view generation of the specific type of vehicle. Training a specific LoRA model can use a smaller training set, thereby reducing training costs and making it easier to mix with other types of models for creative generation, reducing the number of LoRA models actually used, greatly avoiding the noise problem caused by mixing LoRA models, and achieving the one-time generation of images from three perspectives of a certain type of vehicle.

[0102] The embodiment of the present application is a method for generating three views of a car based on multi-view stitching training. By collecting three-view images of the vehicle to construct a training set, the LoRA model is trained. Then, the FLUX model is combined with the trained LoRA model to generate three-view images of the same car. This allows the three views of the same vehicle to be generated at one time when drawing a car design drawing, and the consistency between the three views is guaranteed. The generated images are of high quality and rich in details, providing users with an efficient and convenient solution for generating three views of the vehicle, and also bringing an innovative design model to the automotive design industry. Furthermore, the three-view generation of a specific type of vehicle can be achieved through training with a specific training set, effectively improving the efficiency of generating three views of the vehicle.

[0103] In a second aspect, an embodiment of the present application also provides a device for generating three-view images of a car based on multi-view stitching training.

[0104] In one embodiment, referring to Figure 2 , Figure 2 This is a functional module diagram of the vehicle three-view image generation device based on multi-view stitching training in this application. Figure 2 As shown, the vehicle three-view image generation device based on multi-view stitching training includes: a construction module and an execution module.

[0105] The construction module is used to collect three-view images of the vehicle to construct an image set, and label the image set to obtain a training set; the execution module is used to train the LoRA model based on the training set, and adjust the FLUX model based on the trained LoRA model, so as to realize the three-view generation of the target vehicle based on the adjusted FLUX model.

[0106] On the third aspect, an embodiment of the present application provides a device for generating three views of a car based on multi-perspective stitching training. The device for generating three views of a car based on multi-perspective stitching training can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0107] Reference Figure 3 , Figure 3 This is a hardware structure diagram of a device for generating three-view images of a car based on multi-view stitching training in an embodiment of the present application. In the embodiment of the present application, the device for generating three-view images of a car based on multi-view stitching training may include a processor, a memory, a communication interface, and a communication bus.

[0108] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0109] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect components within the multi-view stitching training-based vehicle three-view image generation device, as well as interfaces used to interconnect the multi-view stitching training-based vehicle three-view image generation device with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet, fiber, or ATM interfaces; user devices can be displays, keyboards, and other devices.

[0110] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0111] The processor may be a general-purpose processor that can call a program for generating three-view images of a vehicle based on multi-view stitching training stored in a memory and execute the method for generating three-view images of a vehicle based on multi-view stitching training provided in the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the program for generating three-view images of a vehicle based on multi-view stitching training is called can refer to the various embodiments of the method for generating three-view images of a vehicle based on multi-view stitching training of the present application, and will not be repeated here.

[0112] Those skilled in the art will understand that Figure 3 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0113] The terms "including" and "having," and any variations thereof, in the specification and claims of this application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus. The terms "first," "second," and "third" are used to distinguish between different objects, etc., and do not indicate a sequential order, nor do they limit the "first," "second," and "third" to different types.

[0114] In the description of the embodiments of this application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0115] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0116] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0118] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.< / lora>

Claims

1. A method for generating three-view images of a car based on multi-view stitching training, characterized in that: The method for generating three-view images of a car based on multi-view stitching training includes: Collecting three-view images of the vehicle to construct an image set, and labeling the image set to obtain a training set; The LoRA model is trained based on the training set, and the FLUX model is adjusted based on the trained LoRA model, so as to realize three-view generation of the target vehicle based on the adjusted FLUX model.

2. The method for generating three-view images of a car based on multi-view stitching training according to claim 1, characterized in that: The collecting of three-view images of the vehicle to construct an image set and labeling the image set to obtain a training set specifically includes: Acquire vehicle images, where the acquired vehicle images include multiple vehicle models, and each vehicle model includes three views; With the vehicle body as the center, the three-view images of the same vehicle model are cropped, and each image is cropped to a preset ratio; The cropped three-view images of the same vehicle model are stitched together in sequence to form a vertically long image, thereby obtaining vertically long images of all vehicle models and constructing an image set.

3. The method for generating three-view images of a car based on multi-view stitching training according to claim 2, characterized in that: The labeling process of the image set to obtain a training set specifically includes: The trigger words are set as text prompts, and the pictures in the image set are labeled to obtain the training set.

4. The method for generating three-view images of a car based on multi-view stitching training according to claim 1, characterized in that: The training of the LoRA model based on the training set specifically includes: Use the FLUX model as the basic model for model training; Build the LoRA model, set up the LoRA model according to the cross attention layer of the FLUX model as needed, and complete the initialization loading of the LoRA model; The hyperparameters are set, and the LoRA model is trained using the training set to obtain a trained LoRA model.

5. The method for generating three-view images of a car based on multi-view stitching training according to claim 4, characterized in that: The training parameters of the LoRA model specifically include: The rank dimension is 64, the scaling factor is 64, the matrix rank random dropout rate is 0.18, the learning rate decay scheduler type is cosine_with_restarts, the basic learning rate is 1e-4, the optimizer type is AdamW8bit, the number of gradient accumulation steps to achieve the maximum equivalent batch is 2, the number of warm-up steps is 440, the maximum training cycle is 16, and the number of samples for a single calculation is 1.

6. The method for generating three-view images of a car based on multi-view stitching training according to claim 1, characterized in that: The method of adjusting the FLUX model based on the trained LoRA model, thereby generating three views of the target vehicle based on the adjusted FLUX model, specifically includes: Setting text prompt words, wherein the keywords in the text prompt words include vehicle type information and environmental information; Setting parameters, including sampling method, number of iteration steps, width and height of latent space, and prompt word guidance coefficient; The FLUX model is adjusted based on the trained LoRA model, so that the FLUX model integrated with the LoRA model executes the generation process according to the text prompt words to realize the three-view generation of the target vehicle.

7. The method for generating three-view images of a car based on multi-view stitching training according to claim 6, characterized in that: The FLUX model is adjusted based on the trained LoRA model, specifically including: The latent space features are iteratively corrected through the trained LoRA model, and the geometric continuity of the three views during stitching is ensured through implicit projection transformation. The geometric shapes and positional relationships under different perspectives or in different states remain continuous and consistent, and the semantic consistency is maintained.

8. The method for generating three-view images of a car based on multi-view stitching training according to claim 1, characterized in that: The method for generating three-view images of a car further comprises: Three-view images of a specific type of vehicle are collected to construct an image set for labeling to obtain a specific training set. The specific training set is used to train the LoRA model to obtain a trained specific LoRA model. The FLUX model is adjusted based on the specific LoRA model, thereby realizing three-view generation of the specific type of vehicle based on the adjusted FLUX model.

9. A device for generating three-view images of a car based on multi-view stitching training, characterized in that: The vehicle three-view image generation device based on multi-view stitching training includes: A construction module is used to collect three-view images of the vehicle to construct an image set, and label the image set to obtain a training set; An execution module is used to train the LoRA model based on the training set, and adjust the FLUX model based on the trained LoRA model, so as to realize three-view generation of the target vehicle based on the adjusted FLUX model.

10. A device for generating three-view images of a car based on multi-view stitching training, characterized in that: The automobile three-view generation device based on multi-view stitching training includes a processor, a memory, and a automobile three-view generation program based on multi-view stitching training stored on the memory and executable by the processor, wherein when the automobile three-view generation program based on multi-view stitching training is executed by the processor, the steps of the automobile three-view generation method based on multi-view stitching training as described in any one of claims 1 to 8 are implemented.

Images