Method and device for generating multi-scene vehicle pictures based on computer vision and medium

Abstract

The invention discloses a method and equipment for generating a multi-scene vehicle picture based on computer vision, and a medium. The method comprises the following steps: obtaining a preset target vehicle picture; performing picture cleaning, picture slicing and picture marking operation on a preset target vehicle picture, and constructing a vehicle picture data set; training a computer vision model by using the constructed vehicle picture data set to obtain a trained computer vision model; receiving a scene prompt word input by a user; and obtaining a target vehicle picture with a target scene through the trained computer vision model. The advanced image recognition and processing technology and the computer vision algorithm are utilized, the single vehicle picture can be automatically converted into the pictures of the vehicle in various different scenes, the labor cost and the transportation cost of on-site shooting are reduced, the manufacturing efficiency of the vehicle propaganda picture is improved, and the manufacturing period is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of image generation, and in particular to a method, device and medium for generating multi-scene vehicle images based on computer vision. Background Art

[0002] With the rapid development of the vehicle industry, manufacturers and various vehicle marketing platforms and media are facing unprecedented challenges - how to efficiently and accurately display the appearance design and performance characteristics of vehicles in different application scenarios. For potential consumers, before making a decision to buy a vehicle, they often hope to obtain detailed information about the target model through multiple channels, especially the performance of the vehicle in various typical usage scenarios, such as busy urban traffic, quiet country roads, high-speed highways, and the driving experience of the target vehicle under complex weather conditions such as rain and snow. In addition, vehicle media and sales agencies also have a strong demand for rich and diverse promotional materials. These materials should not only be able to fully reflect the driving characteristics of the vehicle in various different scenarios, but also have attractive visual effects, so as to better attract consumers' attention and stimulate their desire to buy.

[0003] However, although traditional photography and image editing techniques can produce high-quality vehicle images, these methods have obvious limitations. On the one hand, the vehicle photography process requires a lot of time and resources, from selecting a suitable shooting location to post-processing, each step is costly; on the other hand, the high cost makes it difficult to adapt to the rapidly changing market demand, especially when it is necessary to promote new products or models of vehicles on a large scale, traditional methods are particularly inadequate. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method, device and medium for generating multi-scene vehicle images based on computer vision.

[0005] A first aspect of the present invention provides a method for generating multi-scene vehicle images based on computer vision, comprising the following steps:

[0006] Get the preset target vehicle image;

[0007] Perform image cleaning, image slicing and image labeling operations on the preset target vehicle images to build a vehicle image dataset;

[0008] The computer vision model is trained using the constructed vehicle image dataset to obtain a trained computer vision model;

[0009] receiving a scene prompt word input by a user, wherein the scene prompt word is used to prompt a target scene;

[0010] A target vehicle image with a target scene is obtained by completing the trained computer vision model.

[0011] Furthermore, the preset vehicle image includes a transparent image and a solid image of the target vehicle; the image cleaning operation includes image deduplication, image filtering and image repair.

[0012] Furthermore, the picture slicing operation specifically includes the following steps:

[0013] Establish resolution constraints;

[0014] The preset target vehicle image is segmented into images with multiple resolution sizes according to the preset resolution.

[0015] Furthermore, the image marking operation specifically includes the following steps:

[0016] Delimiting a background area and a vehicle area in the vehicle image, and determining a boundary between the background area and the vehicle area; the vehicle area includes the target vehicle, and the background area includes a scene where the target vehicle is located;

[0017] In the vehicle area, the vehicle is marked with features and categories; the feature marking is used to mark the vehicle appearance features, and the category marking is used to mark the vehicle category;

[0018] In the background area, the scene in which the target vehicle is located is marked; the scene marking is used to mark the scene category in which the target vehicle is located in the vehicle picture;

[0019] Multi-label annotation is performed on the boundary between the background area and the vehicle area; the multi-label annotation is used to separately annotate the interaction between the vehicle and the scene in which it is located.

[0020] Furthermore, the computer vision model is an LDM model, and the LDM model includes a text encoder, a VAE encoder, a noise scheduling module, a U-Net neural network, and a VAE decoder;

[0021] The text encoder is used to receive scene prompt words and encode the scene prompt words into text embedding vectors;

[0022] The VAE encoder is used to receive a target vehicle image and compress the target vehicle image into a low-dimensional space representation;

[0023] The noise scheduling module is used to add random noise to the low-dimensional space representation;

[0024] The U-Net neural network is used to reconstruct the image according to the low-dimensional space representation, and gradually generate a target vehicle picture with a target scene;

[0025] The VAE decoder is used to decode and output the target vehicle picture with the target scene generated by the U-Net neural network.

[0026] Furthermore, the training of the computer vision model specifically includes:

[0027] The compression efficiency of the VAE encoder for the target vehicle image in the vehicle image dataset is determined by the VQ regularization loss function, and the compression efficiency of the VAE encoder is adjusted to the optimum;

[0028] The image reconstruction of the U-Net neural network is controlled by combining the mean square error function with Bayesian statistics, and the image reconstruction effect of the U-Net neural network is adjusted to the optimal value.

[0029] Furthermore, the scene prompt words include angle prompt words, light and shadow prompt words and size prompt words; the angle prompt words are used to prompt the shooting angle of the target scene; the light and shadow prompt words are used to prompt the target light distribution and target shadow distribution in the target scene; the size prompt words are used to prompt the target size of the generated target vehicle picture with the target scene.

[0030] Furthermore, obtaining a target vehicle image having a target scene by completing the trained computer vision model specifically includes the following steps:

[0031] Determine the target vehicle, call the vehicle image dataset of the target vehicle and load it into the computer vision model through the VAE encoder;

[0032] Input the scene prompt words into the computer vision model through the text encoder;

[0033] The VAE decoder obtains a target vehicle picture with a target scene.

[0034] A second aspect of the present invention discloses an electronic device, comprising a processor and a memory;

[0035] The memory is used to store programs;

[0036] The processor executes the program to implement the method of generating multi-scene vehicle images based on computer vision.

[0037] A third aspect of the present invention is a computer-readable storage medium, characterized in that the storage medium stores a program, and the program is executed by a processor to implement the method of generating multi-scene vehicle images based on computer vision.

[0038] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0039] The embodiments of the present invention have the following beneficial effects: the method, device and medium for generating multi-scene vehicle pictures based on computer vision provided by the present invention utilize advanced image recognition, processing technology and computer vision algorithms to automatically convert a single vehicle picture into pictures of the vehicle in a variety of different scenes, thereby reducing the labor cost and transportation cost of on-site shooting, improving the production efficiency of vehicle promotion pictures, and shortening the production cycle. The present invention is widely used in the promotion and distribution of vehicles by vehicle manufacturers, vehicle media and vehicle dealers.

[0040] Additional aspects and advantages of the present invention will be given in part in the following description and, in part, will be obvious from the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a schematic diagram of the main process of a method for generating multi-scene vehicle images based on computer vision according to an embodiment of the present invention.

[0043] Figure 2 It is one of the schematic diagrams of the preset target vehicle picture in the embodiment of the present invention.

[0044] Figure 3 This is the second schematic diagram of the preset target vehicle picture in the embodiment of the present invention.

[0045] Figure 4 It is a schematic diagram of the computer vision model structure used in the present invention.

[0046] Figure 5 The model of the present invention outputs a target vehicle schematic diagram having a target scene.

[0047] Figure 6 It is a structural schematic diagram of an electronic device of the present invention.

[0048] Figure 7 It is a schematic diagram of the structure of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] In order to improve the efficiency of producing vehicle promotional pictures, the first embodiment of the present invention provides a method for generating multi-scene vehicle pictures based on computer vision, such as Figure 1 As shown, the following steps are included:

[0051] S1. Obtain a preset target vehicle image;

[0052] S2. Perform image cleaning, image slicing and image labeling operations on the preset target vehicle images to build a vehicle image dataset;

[0053] S3. Use the constructed vehicle image dataset to train the computer vision model to obtain a trained computer vision model;

[0054] S4. Receive a scene prompt word input by the user, where the scene prompt word is used to prompt the target scene;

[0055] S5. Obtain a target vehicle image with a target scene by completing the trained computer vision model.

[0056] The embodiment of the present invention uses an advanced computer vision model to quickly generate target vehicle images in various scenes. Vehicle promoters only need to use natural language to input prompt words, and the trained computer vision model can quickly output multiple images of target vehicles in target scenes. The embodiment of the present invention effectively reduces the cost of vehicle promotion and is widely accepted by vehicle manufacturers, vehicle media, and vehicle dealers.

[0057] As a further embodiment, the implementation process of each step of the present invention is discussed in detail below.

[0058] S1. Obtain a preset target vehicle image.

[0059] In the embodiment of the present invention, the preset target vehicle image includes a transparent bottom image and a solid bottom image of the target vehicle. Figure 2 As shown, it includes panoramic pictures of the vehicle at various angles under a blank background, which is used to provide the model with the effect of shooting the vehicle from various camera angles; the solid base picture is as follows Figure 3 As shown, the actual driving effect diagrams of the vehicle in various scenarios such as cities, mountains, forests, and deserts are provided to the model to provide the driving effect of the vehicle in various environmental scenarios.

[0060] Through actual testing, it was found that the training effect of setting both the vehicle transparent background image and the real background image in the vehicle image dataset is better than that of using the real background image or the transparent background image alone for training. Therefore, the computer vision model obtained by training with the preset target vehicle image in the embodiment of the present invention has better generalization learning ability and can optimize the model output of the image effects of the vehicle in various scenarios.

[0061] S2. Perform image cleaning, image slicing and image labeling operations on the preset target vehicle images to build a vehicle image dataset.

[0062] In the embodiment of the present invention, image cleaning of the target vehicle image can improve the accuracy of image recognition and image processing, and provide important sample features for subsequent training. The cleaning process includes operations such as data deduplication, data filtering, data repair and data enhancement.

[0063] Exemplarily, data deduplication is accomplished by comparing image hash values, feature vectors and other parameters to delete duplicate images with high similarity; data filtering is accomplished by detecting indicators such as clarity, contrast, and color saturation of each target vehicle image to delete images with quality below a certain level; data repair is accomplished by performing operations such as defect filling and high-definition repair on images through a preset image repair algorithm.

[0064] The image slicing operation in step S2 specifically includes the following steps:

[0065] S2-a1. Establish resolution constraints;

[0066] S2-a2. Segment the preset target vehicle image into images with multiple resolution sizes according to the preset resolution.

[0067] In the embodiment of the present invention, the image slicing operation first determines the required size of the vehicle image generated by the model according to the demand side requirements, such as 512×512, 768×768, 1080×1080, etc.; secondly, the resolution constraint is established according to the different image sizes, and the target vehicle image is segmented into images of multiple resolutions based on the resolution constraint, while retaining the features related to the target vehicle on the image. After actual testing, it was found that the image slicing operation on the target vehicle image helps to improve the clarity and visual effect of the model output at different sizes, and training the model with images of different resolution sizes can effectively improve the image quality of the model at different resolutions.

[0068] The image labeling operation in step S2 specifically includes the following steps:

[0069] S2-b1. Delimit the background area and the vehicle area in the vehicle picture, and determine the boundary between the background area and the vehicle area. Before the image is marked in the embodiment of the present invention, it is first necessary to determine which areas belong to the vehicle area and which areas belong to the background area in the image. The vehicle area refers to the area in the image used to represent the morphological characteristics of the vehicle itself, and the background image is the area in the image used to represent the morphological characteristics of the scene. The embodiment of the present invention uses feature points to calibrate the regional boundaries, and the vehicle area or the background area is formed by connecting several feature points; in another embodiment, the background area and the vehicle area can also be distinguished based on color blocks, relative positions, etc. For example, for target vehicle images of desert scenes or blank scenes, due to their single color tone, the efficiency of calibrating the background area by color blocks is higher than that of calibrating by feature points. At this time, the vehicle area and the background area can be calibrated in the form of color blocks; the distinction based on relative position is mainly applicable to vehicle images with relatively uniform positions of target vehicles in the image. The embodiment of the present invention can help the model learn how to identify vehicles and their features, how to identify scenes and their features, and how to effectively distinguish vehicles and scenes in images by dividing the regions in the vehicle images.

[0070] S2-b2. In the vehicle area, feature annotation and category annotation are performed on the vehicle; feature annotation is used to annotate the vehicle appearance features, and category annotation is used to annotate the vehicle category. In the embodiment of the present invention, feature annotation refers to annotating specific features of the target vehicle. For vehicles of different models, there are differences in their body lines, front face design, wheel design, etc.; for example, racing cars often have streamlined body lines to reduce wind resistance, while SUVs may have more square lines, and the number and opening method of doors (such as side doors or gull-wing doors), wheel style and size are also an aspect of distinguishing different vehicles. Feature annotation of vehicles of different models can provide richer training information, help the model better restore the vehicle image, and avoid missing the key features of the target vehicle. Annotating the category of the vehicle helps the model learn the features of different models, such as sedans, SUVs, trucks, buses, racing cars, etc.

[0071] S2-b3. In the background area, the scene in which the target vehicle is located is annotated; scene annotation is used to annotate the scene category in which the target vehicle is located in the vehicle image. For a transparent image without a scene, scene annotation is not performed. In the embodiment of the present invention, scene annotation is performed on different background areas to help the model understand the driving performance of the vehicle in different scenes.

[0072] S2-b4. Perform multi-label annotation on the boundary between the background area and the vehicle area; multi-label annotation is used to separately annotate the interaction between the vehicle and the scene in which it is located.

[0073] Multi-label annotation helps the model understand the content of the interaction when the vehicle interacts deeply with the scene. For example, when a vehicle is driving in a snowy environment, there will be snow on the body of the vehicle. At this time, multi-label annotation of the snow can help the model identify and process the snow on the body of the vehicle, restoring the true appearance of the target vehicle.

[0074] In the embodiment of the present invention, the vehicle labeling operation is performed through word vectors, and the test results show that the training effect of the training set based on word vector labels is better than that of the training set based on sentence labels.

[0075] S3. Use the constructed vehicle image dataset to train the computer vision model to obtain a trained computer vision model.

[0076] The computer vision model used in the embodiment of the present invention is the LDM (Latent Diffusion Model) model, and the model structure is as follows: Figure 4 As shown. The LDM model is a latent diffusion model that can slowly restore the target image through latent diffusion based on the prompt word, and is suitable for generating a target vehicle image in a target scene. In some other embodiments, other diffusion models such as the DDIM model, the GLDM model, etc. can also be used.

[0077] The results of the LDM model used in the embodiment of the present invention are further described below: The LDM model in the embodiment of the present invention includes a text encoder, a VAE encoder, a noise scheduling module, a U-Net neural network, and a VAE decoder. The text encoder is used to receive scene prompt words and encode the scene prompt words into text embedding vectors; the VAE encoder is used to receive target vehicle pictures and compress the target vehicle pictures into low-dimensional space representations; the noise scheduling module is used to add random noise to the low-dimensional space representation; the U-Net neural network is used to reconstruct images based on the low-dimensional space representation, and gradually generate target vehicle pictures with target scenes; the VAE decoder is used to decode and output the target vehicle pictures with target scenes generated by the U-Net neural network.

[0078] Exemplarily, in the pixel space of the model, the text encoder can use a natural language model such as the CLIP model to convert the input prompt word into a high-dimensional text embedding vector, which captures the semantic information of the prompt word text. The VAE encoder (variational autoencoder) can compress the target vehicle image into a low-dimensional latent space representation; to reduce the amount of calculation, while removing the imperceptible details in the image. In the latent space, the embodiment of the present invention performs a diffusion process through a denoising U-Net neural network. This process gradually recovers the details of the image from the noise, starting from pure noise, and gradually generates the target vehicle image. At the same time, the embodiment of the present invention injects the text embedding vector obtained by the prompt word into the U-Net network through the cross-attention mechanism during the denoising process, so that the model can be diffused based on the prompt word when executing the denoising process, ensuring that the picture generated by the model matches the prompt word description. Through repeated denoising optimization in the latent space, the noise of the picture is gradually reduced, the details of the target vehicle and the scene are becoming clearer and clearer, and finally a target vehicle picture with a target scene consistent with the prompt word description is generated. Finally, the picture in the latent space is converted back to the pixel space through the VAE decoder, and the target vehicle picture with the target scene is output.

[0079] The training of the LDM model of the embodiment of the present invention is divided into two parts. For the pixel space part, the embodiment of the present invention adopts the VQ regularized loss function for training. The VQ regularized loss function can quantize the continuous potential representation vector into a discrete embedding vector. This process involves a codebook, which contains a finite set of embedding vectors. For a given input, the VQ layer finds the embedding vector closest to the input vector in the codebook and replaces the original continuous vector with it. Since the VQ operation itself is not differentiable, STE is required to estimate the gradient during back propagation. STE performs quantization operations during forward propagation and passes the gradient directly to the output of the encoder during back propagation, as if the quantization operation is differentiable. VQ regularization helps prevent the model from learning overly complex representations, thereby avoiding overfitting and improving the generalization ability of the model. Discrete embedding vectors can provide more meaningful feature representations, which helps the model capture the semantic information of the data. For the latent space part, the embodiment of the present invention uses the mean square error (MSE) as a measure of the difference between the noise predicted by the model and the actual noise added. The MSE calculation formula is: Among them, Y i is the real noise, is the noise predicted by the model, and n is the number of samples. In the diffusion process, the model adds Gaussian noise to the image to simulate the transition of data from an ordered state to a disordered state. In the reverse process, the model combines the Bayesian statistical principle to predict the denoised image at each step.

[0080] S4. Receive a scene prompt word input by the user, where the scene prompt word is used to prompt a target scene.

[0081] In the embodiment of the present invention, in addition to describing the scene-related content, the scene prompt words also include angle prompt words, light and shadow prompt words, and size prompt words; the angle prompt words are used to prompt the shooting angle of the target scene; the light and shadow prompt words are used to prompt the target light distribution and target shadow distribution in the target scene; the size prompt words are used to prompt the target size of the target vehicle picture generated with the target scene. In the embodiment of the present invention, the constraints on the scene such as angle, light and shadow, size, etc. are input into the model in the form of prompt words, which helps the model generate pictures with specific shooting angles, light and shadow distributions, and sizes.

[0082] S5. Obtain a target vehicle image with a target scene by completing the trained computer vision model.

[0083] In step S5, a target vehicle image with a target scene is obtained by completing the trained computer vision model, which specifically includes the following steps:

[0084] S5-1. Determine the target vehicle, call the vehicle image dataset of the target vehicle and load it into the computer vision model through the VAE encoder. Step S5-1 is used to determine the target vehicle generated by the model. The embodiment of the present invention constructs a target vehicle image dataset based on different vehicle models. By selecting the target vehicle, the model can call the target vehicle dataset for targeted training and generate scene images related to the target vehicle.

[0085] S5-2. Input the scene prompt words into the computer vision model through the text encoder. The scene prompt words describe the scene, light and shadow, size, style, angle and other contents required by the user through natural language, so that the model can establish scene constraints and generate pictures.

[0086] S5-3. The VAE decoder obtains a target vehicle image with a target scene. The schematic diagram of the target vehicle image with a target scene generated by the model is as follows: Figure 5 After selecting the target vehicle and entering the scene prompt word, the model will generate a target vehicle image with the target scene based on the existing training data of the target vehicle. The vehicle in the image has the specific features of the target vehicle and meets the scene atmosphere required by the scene prompt word, effectively improving the energy efficiency and richness of the image output.

[0087] The embodiments of the present invention can significantly reduce the cost of producing vehicle image materials and greatly improve the efficiency and richness of image output. By introducing automation technology, the present invention greatly reduces the traditional shooting and post-editing work that requires a lot of manpower, thereby achieving significant results in cost control. This improvement not only reduces the consumption of human resources, but also shortens the time cycle from creative conception to finished product output.

[0088] In addition, the present invention innovatively adopts a natural language-driven image generation function, allowing users to guide the image generation process through simple text prompts. This method greatly simplifies the operation process, allowing even non-professional users to easily generate high-quality images that meet specific needs. The application of natural language processing technology not only improves the flexibility and diversity of image generation, but also ensures that the generated images can better meet the user's personalized needs and creative imagination.

[0089] Finally, the invention achieves faster image generation through automated processes and natural language processing while maintaining high quality and relevance. The application of this technology can quickly generate a large number of vehicle images in multiple scenarios without sacrificing image quality.

[0090] Figure 6 : is a schematic diagram of the structure of the electronic device proposed in the second embodiment of the present invention. The memory of this embodiment stores program instructions for implementing the method of generating multi-scene vehicle images based on computer vision in any of the above embodiments. The processor is used to execute the program instructions stored in the memory to generate multi-scene vehicle images based on computer vision. Among them, the processor can also be called a CPU (Central Processing Unit). The processor may be an integrated circuit chip with signal processing capabilities. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0091] The contents of the method in the first embodiment of the present invention are all applicable to the electronic device embodiment. The functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

[0092] Figure 7It is a schematic diagram of the structure of the computer-readable storage medium of the third embodiment of the present invention. The computer-readable storage medium of the fourth embodiment of the present invention stores program instructions that can implement the above-mentioned method of generating multi-scene vehicle pictures based on computer vision, wherein the program instructions can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0093] The contents of the method in the first embodiment of the present invention are all applicable to the computer-readable storage medium embodiment. The functions specifically implemented by the computer-readable storage medium embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

[0094] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the method for generating multi-scene vehicle images based on computer vision provided in the above embodiment.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0096] Those skilled in the art will appreciate that the modules in the devices in the embodiments of the present invention may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments of the present invention may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including corresponding claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including corresponding claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0097] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0098] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0099] In addition, each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. In particular, for embodiments such as devices and equipment, since they are basically similar to method embodiments, the relevant parts can refer to the partial description of the method embodiments. The embodiments of the devices and equipment described above are merely schematic, wherein the modules, units, etc. described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed in multiple places, such as nodes of a system network. Specifically, some or all of the modules and units may be selected according to actual needs to achieve the purpose of the above-mentioned embodiment scheme. Those skilled in the art can understand and implement it without paying creative labor.

[0100] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0101] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0102] In addition, the terms "first", "second", etc. used in the embodiments of the present invention are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present invention can explicitly or implicitly indicate that the embodiment includes at least one of the features. In the description of the present invention, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0103] In the embodiments of the present invention, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present invention may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context in the specific embodiment.

[0104] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and cannot be construed as limitations of the present invention, and those of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention. Those skilled in the art will readily come to think of other embodiments of the present invention after considering the specification and practicing the present invention. This application is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art that are not disclosed in the present invention. The specification and embodiments are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

Claims

1. A method for generating multi-scene vehicle images based on computer vision, characterized in that: The following steps are involved: Get the preset target vehicle image; Perform image cleaning, image slicing and image labeling operations on the preset target vehicle images to build a vehicle image dataset; The computer vision model is trained using the constructed vehicle image dataset to obtain a trained computer vision model; receiving a scene prompt word input by a user, wherein the scene prompt word is used to prompt a target scene; The target vehicle image with the target scene is obtained by completing the trained computer vision model.

2. The method for generating multi-scene vehicle images based on computer vision according to claim 1, characterized in that: The preset vehicle image includes a transparent image and a solid image of the target vehicle; the image cleaning operation includes image deduplication, image filtering and image repair.

3. The method for generating multi-scene vehicle images based on computer vision according to claim 1, characterized in that: The picture slicing operation specifically includes the following steps: Establish resolution constraints; The preset target vehicle image is segmented into images with multiple resolution sizes according to the preset resolution.

4. The method for generating multi-scene vehicle images based on computer vision according to claim 1, characterized in that: The image marking operation specifically includes the following steps: Delimiting a background area and a vehicle area in the vehicle image, and determining a boundary between the background area and the vehicle area; the vehicle area includes the target vehicle, and the background area includes a scene where the target vehicle is located; In the vehicle area, the vehicle is marked with features and categories; the feature marking is used to mark the vehicle appearance features, and the category marking is used to mark the vehicle category; In the background area, the scene in which the target vehicle is located is marked; the scene marking is used to mark the scene category in which the target vehicle is located in the vehicle picture; Multi-label annotation is performed on the boundary between the background area and the vehicle area; the multi-label annotation is used to separately annotate the interaction between the vehicle and the scene in which it is located.

5. The method for generating multi-scene vehicle images based on computer vision according to claim 1, characterized in that: The computer vision model is an LDM model, which includes a text encoder, a VAE encoder, a noise scheduling module, a U-Net neural network and a VAE decoder; The text encoder is used to receive scene prompt words and encode the scene prompt words into text embedding vectors; The VAE encoder is used to receive a target vehicle image and compress the target vehicle image into a low-dimensional space representation; The noise scheduling module is used to add random noise to the low-dimensional space representation; The U-Net neural network is used to reconstruct the image according to the low-dimensional space representation, and gradually generate a target vehicle picture with a target scene; The VAE decoder is used to decode and output the target vehicle picture with the target scene generated by the U-Net neural network.

6. The method for generating multi-scene vehicle images based on computer vision according to claim 5, characterized in that: The training of the computer vision model specifically includes: The compression efficiency of the VAE encoder for the target vehicle image in the vehicle image dataset is determined by the VQ regularization loss function, and the compression efficiency of the VAE encoder is adjusted to the optimum; The image reconstruction of the U-Net neural network is controlled by combining the mean square error function with Bayesian statistics, and the image reconstruction effect of the U-Net neural network is adjusted to the optimal value.

7. The method for generating multi-scene vehicle images based on computer vision according to claim 1, characterized in that: The scene prompt words include angle prompt words, light and shadow prompt words and size prompt words; the angle prompt words are used to prompt the shooting angle of the target scene; the light and shadow prompt words are used to prompt the target light distribution and target shadow distribution in the target scene; the size prompt words are used to prompt the target size of the generated target vehicle picture with the target scene.

8. The method for generating multi-scene vehicle images based on computer vision according to claim 5, characterized in that: The step of obtaining a target vehicle image having a target scene by completing the training of a computer vision model specifically includes the following steps: Determine the target vehicle, call the vehicle image dataset of the target vehicle and load it into the computer vision model through the VAE encoder; Input the scene prompt words into the computer vision model through the text encoder; The VAE decoder obtains a target vehicle picture with a target scene.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement a method for generating multi-scene vehicle images based on computer vision as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement a method for generating multi-scene vehicle images based on computer vision as described in any one of claims 1 to 8.

Images