Virtual image synthesis method, device, equipment and storage medium

Abstract

The present invention relates to the field of virtual imaging technology, and in particular to a virtual image synthesis method, apparatus, device, and storage medium. The method comprises: determining environmental rendering parameters of a preset virtual environment, initial color information of a photographed object, and position parameters relative to the photographed object; adjusting the hue of the photographed object based on the environmental rendering parameters, the initial color information of the photographed object, and the position parameters using an adjustment formula of a preset neural network model; and synthesizing the photographed object with the adjusted hue with the preset virtual environment to obtain a virtual image. The present invention achieves dynamic hue matching between the photographed object and its virtual environment through a preset neural network model, thereby solving the problem that in virtual photography, the environment may have different lighting conditions, color temperatures, and shadow effects, while the hue of the photographed object is often fixed and difficult to automatically adapt to the changing environment.

Description

Technical Field

[0001] The present invention relates to the field of virtual image technology, and in particular to a virtual image synthesis method, device, equipment and storage medium. Background Art

[0002] In the field of virtual photography, technology for placing subjects in diverse virtual environments is developing rapidly. However, a current technical challenge is automatically adjusting the subject's color tone to match these diverse environments. This is because the environments in virtual photography may have varying lighting conditions, color temperatures, and shadow effects, while the subject's color tone is often fixed and difficult to automatically adapt to these changing environments.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a virtual image synthesis method, device, equipment and storage medium, aiming to solve the existing technical problem in the field of virtual photography that when the subject is placed in different virtual environments, the color tone of the subject cannot be automatically adjusted to match the environment.

[0005] To achieve the above object, the present invention provides a virtual image synthesis method, the method comprising:

[0006] Determining environmental rendering parameters of a preset virtual environment, initial color information of a photographed object, and position parameters relative to the photographed object;

[0007] Adjusting the hue of the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter;

[0008] synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image;

[0009] The preset neural network model is obtained by the following steps:

[0010] Acquire image data of objects photographed in different environments;

[0011] Determining environmental background parameters of the image data in a corresponding environment;

[0012] A preset neural network model is obtained by training a neural network using the environmental background parameters.

[0013] Optionally, the adjustment formula of the preset neural network model is:

[0014] f(C,E,P)=σ(Wn*σ(...σ(W1*[C,E,P]+b1)...)+bn);

[0015] Where f(C, E, P) is the result obtained by processing the original color tone through the preset neural network model. f(C, E, P) contains the structure of a multi-layer perceptron or a convolutional neural network. C is the initial color information of the photographed object. E is the environment rendering parameter, which includes various parameters that affect the rendering effect. P is the position parameter of the photographed object, which includes the position of the photographed object in three-dimensional space. σ is the activation function, Wn is the weight of the network, and bn is the bias term of the network.

[0016] Optionally, adjusting the hue of the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter includes:

[0017] Determining a hue adjustment parameter and a hue adjustment area of ​​the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter;

[0018] The hue of the photographed object is adjusted based on the hue adjustment parameter and the hue adjustment area.

[0019] Optionally, the determining the hue adjustment parameter and the hue adjustment area of ​​the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter includes:

[0020] Determining a target detection algorithm and an image segmentation algorithm for the photographed object based on the environment rendering parameters, the initial color information of the photographed object, and the position parameters through an adjustment formula of a preset neural network model;

[0021] The hue adjustment parameters and hue adjustment area of ​​the photographed object are determined based on the target detection algorithm and the image segmentation algorithm.

[0022] Optionally, adjusting the hue of the photographed object based on the hue adjustment parameter and the hue adjustment area includes:

[0023] Determining image pixel values ​​of the photographed object;

[0024] adjusting the image pixel values ​​of the photographed object based on the tone adjustment parameter and the tone adjustment area;

[0025] The hue of the photographed object is determined based on the adjusted image pixel values.

[0026] Optionally, synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image includes:

[0027] Segmenting the toned subject from the original background as a foreground object based on a preset deep learning model, and using the preset virtual environment as a background image;

[0028] The foreground object and the background image are synthesized using a synthetic virtual image formula to obtain a virtual image.

[0029] Optionally, the synthetic virtual image formula is:

[0030] I(x,y)=α(x,y)·C′(x,y)+(1-α(x,y))·B(x,y);

[0031] Where I(x, y) is the pixel value of the final synthesized virtual image at the coordinate (x, y), C′(x, y) is the pixel value of the adjusted foreground object at the coordinate (x, y), B(x, y) is the pixel value of the background image at the coordinate (x, y), and α(x, y) is the transparency of the foreground image, which ranges from [0, 1].

[0032] In addition, to achieve the above-mentioned object, the present invention further provides a virtual image synthesis device, the virtual image synthesis device comprising:

[0033] A parameter determination module is used to determine the environment rendering parameters of the preset virtual environment, the initial color information of the photographed object, and the position parameters of the photographed object;

[0034] a hue adjustment module, configured to adjust the hue of the photographed object using an adjustment formula of a preset neural network model based on the environmental rendering parameters, the initial color information of the photographed object, and the position parameters;

[0035] An image synthesis module, configured to synthesize the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image;

[0036] The preset neural network model is obtained by the following steps:

[0037] Acquire image data of objects photographed in different environments;

[0038] Determining environmental background parameters of the image data in a corresponding environment;

[0039] A preset neural network model is obtained by training a neural network using the environmental background parameters.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also provides a virtual image synthesis device, which includes: a memory, a processor, and a virtual image synthesis program stored on the memory and runnable on the processor, and the virtual image synthesis program is configured to implement the steps of the virtual image synthesis method as described in any one of the above items.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a virtual image synthesis program is stored. When the virtual image synthesis program is executed by a processor, the steps of the virtual image synthesis method as described in any one of the above items are implemented.

[0042] The present invention provides a virtual image synthesis method, comprising: determining environmental rendering parameters of a preset virtual environment, initial color information of a photographed object, and position parameters relative to the photographed object; adjusting the hue of the photographed object using an adjustment formula of a preset neural network model based on the environmental rendering parameters, the initial color information of the photographed object, and the position parameters; and synthesizing the hue-adjusted photographed object with the preset virtual environment to obtain a virtual image. The present invention achieves dynamic hue matching between the photographed object and its virtual environment through a preset neural network model, thereby resolving the problem in virtual photography that the environment may have different lighting conditions, color temperatures, and shadow effects, while the hue of the photographed object is often fixed and difficult to automatically adapt to the changing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the structure of a virtual image synthesis device in the hardware operating environment involved in the embodiment of the present invention;

[0044] Figure 2 1 is a flow chart of a first embodiment of a virtual image synthesis method according to the present invention;

[0045] Figure 3 is a flow chart of a second embodiment of a virtual image synthesis method according to the present invention;

[0046] Figure 4 is a flowchart of a third embodiment of a virtual image synthesis method according to the present invention;

[0047] Figure 5 FIG. 1 is a structural block diagram of a first embodiment of a virtual image synthesis device according to the present invention.

[0048] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] Reference Figure 1 , Figure 1 It is a structural diagram of a virtual image synthesis device in a hardware operating environment involved in an embodiment of the present invention.

[0051] like Figure 1 As shown, the virtual image synthesis device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and optionally the user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (RAM) memory, or a stable memory (NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the virtual image synthesis device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0053] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a virtual image synthesis program.

[0054] exist Figure 1 In the virtual image synthesis device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to peripheral devices; the virtual image synthesis device calls the virtual image synthesis program stored in the memory 1005 through the processor 1001, and executes the virtual image synthesis method provided by the embodiment of the present invention.

[0055] Based on the above hardware structure, an embodiment of the virtual image synthesis method of the present invention is proposed.

[0056] Reference Figure 2 , Figure 2FIG1 is a flow chart of a first embodiment of a virtual image synthesis method according to the present invention, and provides a first embodiment of a virtual image synthesis method according to the present invention.

[0057] In a first embodiment, the virtual image synthesis method includes the following steps:

[0058] S10: Determine environmental rendering parameters of a preset virtual environment, initial color information of the photographed object, and position parameters relative to the photographed object.

[0059] It should be noted that the execution entity of this embodiment can be a software system or platform. This system can integrate multiple technologies, including but not limited to image processing software, computer vision libraries and deep learning frameworks, or other electronic devices that can achieve the above functions. This embodiment does not limit this.

[0060] It should be noted that a pre-set virtual environment refers to a pre-designed and pre-set digital environment within a virtual image or 3D simulation. These environments are typically created using computer graphics and modeling techniques and incorporate various visual elements and effects to simulate the real world or create a specific artistic style. The rendering parameters of a pre-set virtual environment typically encompass multiple aspects, defining and controlling the visual presentation within the virtual environment to make it as close to the real world as possible or conform to a specific artistic style. These parameters include, but are not limited to, lighting settings, material properties, shadows and reflections, color schemes, environmental effects, viewing angles, and field of view. The initial color information of the photographed object refers to the original color characteristics of the object before it is placed in the virtual environment. This original color characteristics include color composition, material properties, and texture information. This initial color information provides important baseline data for the neural network model. Based on this information and incorporating the characteristics of the virtual environment, the model can calculate how the object should appear in the new environment. Preserving this original information also helps ensure that the adjusted object retains its essential characteristics and recognizability, while naturally integrating into the new environment.

[0061] It should be noted that the subjects being photographed can be diverse, including real people, objects, animals and plants, etc. The position parameters of the subject being photographed refer to a set of data that describes the specific position and posture of the subject in the photographic space. The data may include: position parameters, rotation parameters, scaling parameters, posture parameters, speed and acceleration, anchor points, paths, etc. These position parameters can be captured by various sensors and devices (such as cameras, lidar, inertial measurement units, etc.) and used in post-processing or real-time processing to ensure the logical relationship and physical consistency between the subject being photographed and the virtual background.

[0062] S20: Adjusting the color tone of the photographed object through an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter.

[0063] It should be noted that the preset neural network model refers to a neural network model that has been completed through a predefined architecture and training process. Such a model has been fully trained and verified and can be directly applied to specific tasks. The hue of the subject refers to the color attribute of the object in the image, including its color type (such as red, green, blue) and the depth of the color. Adjusting the hue of the subject involves changing its color performance to make it more coordinated with the ambient lighting and surrounding background to obtain a more natural visual effect.

[0064] In the specific implementation, first obtain an original image (photo or video frame) containing the subject and the ambient lighting conditions, including the light source's color (color temperature), direction, and intensity. Obtain information about the surrounding background color and material, as this information influences the final effect of the color adjustment. Extract the subject's position parameters in the image, including its outline, bounding box, and other information. This can be achieved using object detection algorithms (such as YOLO and SSD) or segmentation algorithms (such as Mask R-CNN). Select a neural network model suitable for the color adjustment task, such as one based on a convolutional neural network (CNN) or a generative adversarial network (GAN). Pretrain the model on a large dataset of images containing different lighting and environmental conditions so that it can learn various lighting and color variations. Load the pretrained neural network model into memory in preparation for inference. Normalize the image data to meet model input requirements (e.g., resize and normalize). Format the ambient rendering parameters and position parameters into the form required by the model, for example, by converting the lighting information into a numerical feature vector. Input the normalized image, formatted ambient rendering parameters, and position parameters into the pre-set neural network model. The model infers the input data and outputs hue adjustment suggestions, including adjustment values ​​for the object's hue, saturation, and brightness. The desired hue adjustment values ​​are obtained from the model output. Image processing techniques (such as OpenCV and PIL) are used to modify the object's hue based on the hue, saturation, and brightness adjustment values ​​provided by the model. These operations include: Hue adjustment: changing the dominant wavelength of a color. Saturation adjustment: increasing or decreasing the richness of a color. Brightness adjustment: increasing or decreasing brightness to make an object brighter or darker.

[0065] S30: synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image.

[0066] It's important to re-composite the adjusted object onto the original background image to ensure the tonal adjustment blends in with the surroundings. You can verify the results through visual inspection or automated tools to ensure the tonal adjustment is natural and matches the surroundings. If the result is unsatisfactory, fine-tune or iterate until you achieve the desired result.

[0067] Furthermore, in this embodiment, the preset neural network model is obtained by the following steps:

[0068] S01: Acquire image data of objects being photographed in different environments.

[0069] It should be noted that different environments refer to the various parameters and scenarios that affect the appearance of the photographed subject. Image data of the photographed subject refers to images of the subject captured under different environmental conditions. This image data should be able to reflect the actual changes in the subject's appearance under various lighting, background, and angle conditions.

[0070] S02: Determine the environmental background parameters of the image data in the corresponding environment.

[0071] It should be noted that environmental background parameters refer to various characteristics and conditions that describe the environment in which the subject is photographed. In the specific implementation, step 1: read the image and metadata, step 2: extract the lighting parameters, and estimate the light source direction and intensity through convolutional neural networks (CNN) or image histogram analysis, step 3: extract background color and texture features, step 4: infer weather conditions, and infer the weather through the overall hue and brightness of the image, step 5: combine all the parameters and summarize all the extracted parameters.

[0072] S03: The preset neural network model is obtained by training the neural network through the environmental background parameters. In the specific implementation, 1. The environmental parameters are standardized or encoded into a format suitable for model input, 2. A data pipeline is established, and an efficient data pipeline can be established using the TensorFlow dataset (tf.data) API, 3. A model is built, and a multi-input model is built using Keras, 4. The model is compiled, and a suitable optimizer, loss function, and evaluation indicator are selected, 5. The model is trained, and the dataset is input into the model for training, and the performance is monitored using the validation set, 6. Model evaluation, and the model performance is evaluated using the test set to check whether there is overfitting or underfitting, 7. Model optimization, and the model is optimized according to the evaluation results, and the network architecture can be adjusted, regularization technology can be added, and parameters can be adjusted. 8. The model is saved, and the trained model is saved for subsequent use, 9. The model is deployed, and the model is deployed in a production environment, and the model is used for real-time prediction. Through the above steps, the neural network model can be trained using environmental background parameters, thereby improving the robustness and accuracy of the model under different environmental conditions.

[0073] Furthermore, in this embodiment, the adjustment formula of the preset neural network model is:

[0074] f(C,E,P)=σ(Wn*σ(....σ(W1*[C,E,P]+b1)...)+bn);

[0075] Where f(C, E, P) is the result obtained by processing the original color tone through the preset neural network model. f(C, E, P) contains the structure of a multi-layer perceptron or a convolutional neural network. C is the initial color information of the photographed object. E is the environment rendering parameter, which includes various parameters that affect the rendering effect. P is the position parameter of the photographed object, which includes the position of the photographed object in three-dimensional space. σ is the activation function, Wn is the weight of the network, and bi is the bias term of the network.

[0076] It should be noted that C is the initial color information of the photographed object, which includes color value, texture information, lighting information, reflection and refraction information, etc. E is the environmental rendering parameter, which includes various parameters that affect the rendering effect, including light intensity, color temperature, environmental reflectivity, shadow softness, atmospheric scattering, etc. P is the position parameter of the photographed object, which includes the position of the photographed object in three-dimensional space. The position of the photographed object will affect the lighting, shadow and other effects it receives. σ is the activation function, which is a key component in the neural network. It introduces nonlinearity to the network, enabling the network to learn and simulate complex patterns. Wn is the weight matrix of the nth layer of the network, and bn is the bias vector of the nth layer of the network.

[0077] Reference Figure 3 , Figure 3 2 is a flow chart of a second embodiment of the virtual image synthesis method of the present invention. In the second embodiment, adjusting the hue of the photographed object based on the environment rendering parameters and the position parameters through a preset neural network model includes:

[0078] S201: Determine the hue adjustment parameters and hue adjustment area of ​​the photographed object based on the environment rendering parameters, the initial color information of the photographed object, and the position parameters through an adjustment formula of a preset neural network model.

[0079] S202: Adjusting the hue of the photographed object based on the hue adjustment parameter and the hue adjustment area.

[0080] It should be noted that tone adjustment parameters refer to a set of values ​​used to adjust the color properties of an image or image region in image processing. A tone adjustment region refers to the specific area in an image that requires tone adjustment. This region is determined by parameters such as specific coordinates and size, or a specific region selection method.

[0081] In specific implementations, a pre-set neural network model processes input information and outputs the parameters and regions required for tonal adjustment. The system first collects three key pieces of information: environmental rendering parameters, including lighting conditions and atmospheric conditions; initial color information of the captured object, which can be RGB values ​​or data in other color spaces; and position parameters, which represent the object's position within the virtual scene. These input data are standardized or normalized to bring their values ​​into a similar range, facilitating neural network processing. The pre-processed data is then fed into a pre-set neural network model, which can be a multi-layer perceptron (MLP) or a convolutional neural network (CNN). Through multiple layers of nonlinear transformations, the network learns the complex relationship between the input data and the desired tonal adjustment. The neural network output consists of two components: tonal adjustment parameters, which can include adjustment values ​​for multiple dimensions such as brightness, contrast, saturation, and hue; and tonal adjustment regions, which are masks or weight maps indicating the degree of adjustment required for different regions of the image. The system interprets the network output as specific adjustment instructions, such as "increase brightness by 10%" or "increase contrast in the upper left corner of the image." Based on the tonal adjustment parameters, the system performs global adjustments to the image. This can include: Brightness adjustment: brightening or lowering the overall image brightness. Contrast adjustment: enhancing or reducing the contrast between light and dark in the image. Saturation adjustment: increasing or decreasing the vividness of colors. Hue adjustment: fine-tuning the overall hue. Using the hue adjustment area information, the system applies differentiated processing to different parts of the image, using the adjustment area as a mask to determine the degree of adjustment for each pixel. Soft masking techniques can be used to achieve a gradual adjustment effect. The system processes the image pixel by pixel or block by block, combining global and local adjustments: for each pixel, a hue change is applied to the pixel based on its weight within the adjustment area. Interpolation techniques can be used to ensure a smooth transition. After completing the main adjustments, the system performs additional processing to ensure that the adjusted pixel values ​​are within the valid range (e.g., 0-255). Some smoothing or sharpening may be applied to enhance the overall visual effect. The system outputs the adjusted image, appearing intelligently adjusted based on the environment, initial color, and position. This process, combining environmental awareness with intelligent algorithms, achieves adaptive hue adjustment of the virtual image, making the final image visually more natural and harmonious, better suited to the rendering requirements of virtual environments.

[0082] Furthermore, in this embodiment, the determining of the hue adjustment parameter and the hue adjustment area of ​​the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter includes:

[0083] Determining a target detection algorithm and an image segmentation algorithm for the photographed object based on the environment rendering parameters, the initial color information of the photographed object, and the position parameters through an adjustment formula of a preset neural network model;

[0084] The hue adjustment parameters and hue adjustment area of ​​the photographed object are determined based on the target detection algorithm and the image segmentation algorithm.

[0085] It's important to note that object detection algorithms are used to identify and locate objects of interest (such as people, vehicles, and animals) in images, returning bounding boxes and category labels for these objects. Image segmentation algorithms are used to divide images into semantically meaningful regions. Semantic segmentation assigns each pixel to a category, while instance segmentation distinguishes not only between categories but also between instances of the same category. These two techniques can be used in combination to achieve detailed and precise processing of objects in an image.

[0086] Furthermore, in this embodiment, adjusting the hue of the photographed object based on the hue adjustment parameter and the hue adjustment area includes:

[0087] Determining image pixel values ​​of the photographed object;

[0088] adjusting the image pixel values ​​of the photographed object based on the tone adjustment parameter and the tone adjustment area;

[0089] The hue of the photographed object is determined based on the adjusted image pixel values.

[0090] It's important to note that the pixel value of a photographed object refers to the color information of all the pixels that make up that object in the image. Each pixel typically contains multiple channels, such as red, green, and blue (RGB), each representing the intensity of that color component. For color images, each pixel typically consists of three values: (R, G, B), typically ranging from 0 to 255.

[0091] In a specific implementation, first, pixel values ​​of the captured object are obtained from the image. This may involve reading the image and using object detection and image segmentation algorithms to determine the region of interest. Next, the image pixel values ​​are adjusted based on the provided hue adjustment parameters. For example, assume that the hue adjustment parameters include brightness, contrast, and saturation adjustments. Finally, the adjusted pixel values ​​are mapped back to the corresponding positions in the original image, completing the hue adjustment process for the captured object.

[0092] Reference Figure 4 , Figure 42 is a flow chart of a third embodiment of a virtual image synthesis method according to the present invention. In the third embodiment, synthesizing the tonal-adjusted photographed object with the preset virtual environment to obtain a virtual image includes:

[0093] S301: Segmenting the toned object from the original background as a foreground object based on a preset deep learning model, and using the preset virtual environment as a background image.

[0094] S302: Synthesizing the foreground object and the background image using a synthetic virtual image formula to obtain a virtual image.

[0095] The formula for synthesizing virtual images is:

[0096] I(x,y)=α(x,y)·C′(x,y)+(1-α(x,y))·B(x,y);

[0097] Where I(x, y) is the pixel value of the final synthesized virtual image at the coordinate (x, y), C′(x, y) is the pixel value of the adjusted foreground object at the coordinate (x, y), B(x, y) is the pixel value of the background image at the coordinate (x, y), and α(x, y) is the transparency of the foreground image, which ranges from [0, 1].

[0098] It should be noted that a preset deep learning model (such as U-Net, Mask R-CNN, etc.) is used to perform semantic segmentation on the tonal-adjusted subject. This model has been trained to identify and segment the main objects in the image. The segmentation process generates a mask that accurately identifies the outline of the foreground object. Using the generated mask, the foreground object is extracted from the original image. This process retains all the details of the foreground object, including the previously performed tonal adjustments. A preset virtual environment is used as the background image. This virtual environment can be a fully rendered 3D scene or a high-quality picture of the environment.

[0099] It should be understood that the α(x, y) value can come from the output of the segmentation model, indicating the probability that each pixel belongs to the foreground. For the edge area of ​​the foreground object, α(x, y) can have a gradient value to achieve a smooth transition. For each pixel (x, y) in the image, a synthesis formula is applied. When α = 1, the output pixel is completely determined by the foreground. When α = 0, the output pixel is completely determined by the background. Intermediate values ​​will produce a semi-transparent effect, achieving a smooth fusion of the foreground and background. Traverse each pixel position (x, y) of the image. Obtain the foreground pixel value C′(x, y), background pixel value B(x, y) and transparency α(x, y) at that position. Apply the formula I(x, y) = α(x, y)·C′(x, y)+(1-α(x, y))·B(x, y) to calculate the synthesized pixel value. The calculation result is stored in the corresponding position of the output image. In the edge area of ​​the foreground object, feathering or anti-aliasing techniques can be used to improve the edge quality. This process is repeated for the entire image to produce the final composite virtual image. Global adjustments, such as color balancing or brightness equalization, can be made to ensure visual consistency between the foreground and background. Additional effects, such as depth-of-field blur, can be applied to enhance realism. Through this process, the system seamlessly blends the tonal-adjusted foreground object into the pre-set virtual environment, creating a visually coherent and realistic virtual image. This approach allows for foreground object detail to be preserved while naturally blending into the new background, resulting in a high-quality composite image.

[0100] In addition, an embodiment of the present invention further provides a storage medium, on which a virtual image synthesis program is stored. When the virtual image synthesis program is executed by a processor, the steps of the virtual image synthesis method described above are implemented.

[0101] In addition, refer to Figure 5 The embodiment of the present invention further provides a virtual image synthesis device, the virtual image synthesis device comprising:

[0102] A parameter determination module 10 is used to determine the environment rendering parameters of the preset virtual environment, the initial color information of the photographed object, and the position parameters of the photographed object;

[0103] a hue adjustment module 20 for adjusting the hue of the photographed object using an adjustment formula of a preset neural network model based on the environmental rendering parameters, the initial color information of the photographed object, and the position parameters;

[0104] An image synthesis module 30 is configured to synthesize the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image;

[0105] The preset neural network model is obtained by the following steps:

[0106] Acquire image data of objects photographed in different environments;

[0107] Determining environmental background parameters of the image data in a corresponding environment;

[0108] A preset neural network model is obtained by training a neural network using the environmental background parameters.

[0109] Other embodiments or specific implementations of the virtual image synthesis device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0110] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0111] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), a magnetic disk, or an optical disk), and includes a number of instructions for enabling an end-user device (which can be a mobile phone, a computer, a server, an air conditioner, or a network user device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A virtual image synthesis method, characterized in that: The method comprises: Determining environmental rendering parameters of a preset virtual environment, initial color information of a photographed object, and position parameters relative to the photographed object; The environment rendering parameters are used to define and control the visual performance in the virtual environment, and the environment rendering parameters include: lighting settings, material properties, shadows and reflections, color configuration, environment effects, viewing angle and field of view; The initial color information of the photographed object refers to the original color characteristics of the object before it is placed in the virtual environment, including color composition, material characteristics, and texture information; The position parameters of the photographed object refer to a set of data that describes the specific position and posture of the photographed object in the photographic space. The data includes: position parameters, rotation parameters, scale parameters, posture parameters, speed and acceleration, anchor point, and path; Adjusting the hue of the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter; synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image; The preset neural network model is obtained by the following steps: Acquire image data of objects photographed in different environments; Determining environmental background parameters of the image data in a corresponding environment; The environmental background parameters refer to the characteristics and conditions that describe the environment in which the subject is photographed, including images, metadata, lighting parameters, light source direction, intensity, background color, texture characteristics and weather conditions; Training a neural network using the environmental background parameters to obtain a preset neural network model; The adjustment formula of the preset neural network model is: ； Where, This is the result obtained by processing the original color tone through the preset neural network model. The internal structure contains a multi-layer perceptron or convolutional neural network. is the initial color information of the photographed object, It is the environment rendering parameter, including various parameters that affect the rendering effect. is the position parameter of the photographed object, including the position of the photographed object in three-dimensional space, is the activation function, is the weight matrix of the nth layer of the network, is the bias vector of the nth layer of the network; The step of synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image includes: Segmenting the toned subject from the original background as a foreground object based on a preset deep learning model, and using the preset virtual environment as a background image; Synthesizing the foreground object and the background image using a synthetic virtual image formula to obtain a virtual image; The synthetic virtual image formula is: ； Where, For the final synthetic virtual image at coordinates The pixel value at The coordinates of the adjusted foreground object The pixel value at For the background image at coordinates The pixel value at The transparency of the foreground image, ranging from [0, 1].

2. The virtual image synthesis method according to claim 1, wherein: The adjusting the hue of the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter includes: Determining a hue adjustment parameter and a hue adjustment area of ​​the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter; The hue of the photographed object is adjusted based on the hue adjustment parameter and the hue adjustment area.

3. The virtual image synthesis method according to claim 2, wherein: The determining of the hue adjustment parameter and the hue adjustment area of ​​the photographed object by using an adjustment formula of a preset neural network model based on the environment rendering parameter, the initial color information of the photographed object, and the position parameter includes: Determining a target detection algorithm and an image segmentation algorithm for the photographed object based on the environment rendering parameters, the initial color information of the photographed object, and the position parameters through an adjustment formula of a preset neural network model; The hue adjustment parameters and hue adjustment area of ​​the photographed object are determined based on the target detection algorithm and the image segmentation algorithm.

4. The virtual image synthesis method according to claim 3, wherein: The adjusting the hue of the photographed object based on the hue adjustment parameter and the hue adjustment area includes: Determining image pixel values ​​of the photographed object; adjusting the image pixel values ​​of the photographed object based on the tone adjustment parameter and the tone adjustment area; The hue of the photographed object is determined based on the adjusted image pixel values.

5. A virtual image synthesis device, characterized in that: The virtual image synthesis device includes: A parameter determination module is used to determine the environment rendering parameters of the preset virtual environment, the initial color information of the photographed object, and the position parameters of the photographed object; The environment rendering parameters are used to define and control the visual performance in the virtual environment, and the environment rendering parameters include: lighting settings, material properties, shadows and reflections, color configuration, environment effects, viewing angle and field of view; The initial color information of the photographed object refers to the original color characteristics of the object before it is placed in the virtual environment, including color composition, material characteristics, and texture information; The position parameters of the photographed object refer to a set of data that describes the specific position and posture of the photographed object in the photographic space. The data includes: position parameters, rotation parameters, scale parameters, posture parameters, speed and acceleration, anchor point, and path; a hue adjustment module, configured to adjust the hue of the photographed object using an adjustment formula of a preset neural network model based on the environmental rendering parameters, the initial color information of the photographed object, and the position parameters; An image synthesis module, configured to synthesize the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image; The preset neural network model is obtained by the following steps: Acquire image data of objects photographed in different environments; Determining environmental background parameters of the image data in a corresponding environment; The environmental background parameters refer to the characteristics and conditions that describe the environment in which the subject is photographed, including images, metadata, lighting parameters, light source direction, intensity, background color, texture characteristics and weather conditions; Training a neural network using the environmental background parameters to obtain a preset neural network model; The adjustment formula of the preset neural network model is: ； Where, This is the result obtained by processing the original color tone through the preset neural network model. The internal structure contains a multi-layer perceptron or convolutional neural network. is the initial color information of the photographed object, It is the environment rendering parameter, including various parameters that affect the rendering effect. is the position parameter of the photographed object, including the position of the photographed object in three-dimensional space, is the activation function, is the weight matrix of the nth layer of the network, is the bias vector of the nth layer of the network; The step of synthesizing the photographed object after adjusting the color tone with the preset virtual environment to obtain a virtual image includes: Segmenting the toned subject from the original background as a foreground object based on a preset deep learning model, and using the preset virtual environment as a background image; Synthesizing the foreground object and the background image using a synthetic virtual image formula to obtain a virtual image; The synthetic virtual image formula is: ； Where, For the final synthetic virtual image at coordinates The pixel value at The coordinates of the adjusted foreground object are The pixel value at For the background image at coordinates The pixel value at The transparency of the foreground image, ranging from [0, 1].

6. A virtual image synthesis device, characterized in that: The device includes: a memory, a processor, and a virtual image synthesis program stored in the memory and executable on the processor, wherein the virtual image synthesis program is configured to implement the steps of the virtual image synthesis method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores a virtual image synthesis program, which, when executed by a processor, implements the steps of the virtual image synthesis method according to any one of claims 1 to 4.

Images