A method for generating virtual photographic images based on neural network

Abstract

The present invention relates to the field of virtual photographic image technology, and in particular to a method for generating virtual photographic images based on a neural network. By constructing a green screen environment and adopting a multi-angle shooting technology, a depth perception neural network is trained to calculate the depth information of an object block and the position of the object block. The shadow features are extracted using a convolutional neural network, and the depth information, position and real-time lighting parameters of the object block are combined. The BP neural network is used to comprehensively analyze and learn these data to accurately predict the light and shadow effects of each object block under specific lighting conditions, thereby improving the consistency of light and shadow and visual realism. By semantically segmenting the virtual environment, accurately processing the occlusion relationship between objects in the scene, simulating the interaction of light on the surface of an object, achieving light and shadow fusion in overlapping scenes, optimizing the reflective area and color value adjustment of the object block, ensuring the correct rendering of the shading and reflection effects between objects, and achieving the correct rendering of the mutual shading and reflection effects between objects while matching the light and shadow effects in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual photographic images, and in particular to a method for generating virtual photographic images based on a neural network. Background Art

[0002] In the field of digital film and television production and computer graphics, virtual photography technology has gradually become an important tool for creating visual effects. Traditional virtual photography technology relies on complicated physical equipment setup and post-processing to create a visual fusion between real-world scenes and computer-generated virtual environments. This process usually involves the use of green screen technology, where objects or characters are shot in front of a specific green screen background, and then these elements are composited into different backgrounds through post-processing. Although this method is widely used in film production and TV shows, it still faces multiple challenges, especially in terms of light and shadow processing.

[0003] Specifically, since the lighting conditions simulated in the virtual environment will change with the shooting angle or scene during the photography process, it is difficult to fully match the light and shadow effects, which affects the audience's visual experience and perceived authenticity. In complex scenes containing multiple objects, mutual occlusion between objects is particularly important for the correct rendering of light and shadow. Existing technologies often find it difficult to accurately simulate the impact of occlusion between objects on light propagation, especially in scenes with a large number of objects and crowded scenes. Correctly rendering the mutual occlusion and reflection effects between objects is a major challenge. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for generating virtual photographic images based on a neural network.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A method for generating virtual photographic images based on a neural network comprises the following steps:

[0007] S1. Build a green screen environment and shoot green screen environment images under different lighting directions;

[0008] S2, based on a plurality of green screen environment images, extracting block depth information from the green screen environment images through a depth perception neural network, and calculating the relative position of the object block in the green screen environment;

[0009] S3, extracting the shadow of the block based on the convolutional neural network, taking the depth information of the block, the relative position of the block in the green screen environment and the illumination parameters as the input elements of the BP neural network, and taking the shadow of the block as the output element to train the BP neural network;

[0010] S4, generating a virtual environment image based on the green screen environment image matching, performing semantic segmentation based on the virtual environment, and performing material matching for objects in the virtual environment image according to the semantic segmentation result;

[0011] S5. Set the lighting parameters and select the reflection value range according to the material of the objects in the virtual environment, calculate the shadow of the objects based on the trained BP neural network, fuse the shadows of several objects in the virtual environment image, and calculate the reflection area of ​​the objects. Regulate the color value of the objects in the virtual environment image through the reflection area and the reflection value range to generate a virtual photographic image.

[0012] Furthermore, the object blocks include character blocks and scene blocks.

[0013] Furthermore, the S1 comprises the following steps:

[0014] Set up light sources at different angles and positions in the green screen environment;

[0015] Under the condition of a single light source or a combination of light sources, the green screen environment is photographed from several different angles to obtain an image of the green screen environment;

[0016] Each green screen environment image is annotated, including distance information, light source direction, and light intensity.

[0017] Further, the S2 comprises the following steps:

[0018] The laser sensing distance and pixel occupancy information of the object in each image are used as the input of the depth perception neural network, and the depth of the object is used as the output of the depth perception neural network for training;

[0019] The depth of the object is obtained through the depth perception neural network, and the relative position of the object in the green screen environment is calculated based on the depth of the object and the shooting angle.

[0020] Furthermore, the method of extracting the shadow features of the blocks based on the convolutional neural network includes annotating the shadows of the blocks according to the three-point-one-line principle to divide the shadows of individual blocks, and outputting the shadow pixel matrix of the blocks through the convolutional neural network for the annotated blocks.

[0021] Furthermore, the lighting parameters include the relative position of the lighting in the green screen environment, the lighting intensity and the lighting angle.

[0022] Furthermore, the performing semantic segmentation based on the virtual environment and performing material matching for objects in the virtual environment image according to the semantic segmentation result includes the following steps:

[0023] The green screen environment image is processed through the semantic segmentation algorithm to automatically identify and segment different objects and background elements in the image;

[0024] According to the segmentation result, the material of the object is obtained;

[0025] Block-based materials traverse the material table and match corresponding material properties, including surface texture, glossiness, and color.

[0026] Furthermore, the step of fusing the shadows of several objects in the virtual environment image and calculating the reflective area of ​​the objects comprises the following steps:

[0027] Based on the positional relationship, shadow characteristics, and relative height and angle between each object, identify the combination of objects that may cause light occlusion;

[0028] The overlapped images of the identified occluded objects are subjected to weight reduction fusion to generate the final image;

[0029] For the identified occluding blocks, the light propagation model is applied to simulate the path of light from the light source to the surface of each block and determine the illuminated area of ​​each affected block.

[0030] Furthermore, the controlling the color value of the object in the virtual environment image by the reflective area and the reflective value interval includes:

[0031] Applying a dynamic color value adjustment algorithm to adjust the color value of objects in the virtual environment image according to the reflective area and the selected reflective value interval;

[0032] The adjusted color values ​​of the objects are post-processed, including color balancing, contrast optimization, and brightness and darkness adjustment.

[0033] The beneficial effect of the present invention is that the present invention collects depth information about objects and their precise positions in the scene by constructing a green screen environment and using multi-angle shooting technology. These data are used to train a depth perception neural network that can understand the three-dimensional structure and relative position of objects in virtual space. This step provides the necessary basic data for subsequent light and shadow calculation and object occlusion processing. The shadowing features of the blocks are extracted using a convolutional neural network, and the back propagation neural network is used to comprehensively analyze and learn these data in combination with the depth information, position and real-time lighting parameters of the blocks. This enables the system to accurately predict and generate the light and shadow effects of each object under specific lighting conditions, greatly improving the consistency of light and shadow and visual realism. In addition, by semantically segmenting the virtual environment, the occlusion relationship between objects in complex scenes can be accurately identified and processed. The system can simulate the interaction of light on the surfaces of various objects, including reflection, refraction and shadow projection, so as to achieve correct light and shadow fusion between objects in complex scenes. This not only solves the problem of mutual occlusion between objects, but also optimizes the reflective area and color value adjustment of the objects, so as to achieve the correct rendering of mutual occlusion and reflection effects between objects while matching the light and shadow effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the steps of a method for generating virtual photographic images based on a neural network in the present invention.

[0035] Figure 2 It is a flowchart of the steps of performing semantic segmentation based on a virtual environment and performing material matching for objects in a virtual environment image according to the semantic segmentation result in the present invention. DETAILED DESCRIPTION

[0036] See also Figure 1-2 As shown, the present invention relates to a method for generating a virtual photographic image based on a neural network, comprising the following steps:

[0037] S1. Build a green screen environment and shoot green screen environment images under different lighting directions;

[0038] S2, based on a plurality of green screen environment images, extracting block depth information from the green screen environment images through a depth perception neural network, and calculating the relative position of the object block in the green screen environment;

[0039] S3, extracting the shadow of the block based on the convolutional neural network, taking the depth information of the block, the relative position of the block in the green screen environment and the illumination parameters as the input elements of the BP neural network, and taking the shadow of the block as the output element to train the BP neural network;

[0040] S4, generating a virtual environment image based on the green screen environment image matching, performing semantic segmentation based on the virtual environment, and performing material matching for objects in the virtual environment image according to the semantic segmentation result;

[0041] S5. Set the lighting parameters and select the reflection value range according to the material of the objects in the virtual environment, calculate the shadow of the objects based on the trained BP neural network, fuse the shadows of several objects in the virtual environment image, and calculate the reflection area of ​​the objects. Regulate the color value of the objects in the virtual environment image through the reflection area and the reflection value range to generate a virtual photographic image.

[0042] It should be noted that the implementation of step S1 involves constructing a green screen environment and capturing images of the object from multiple angles and under different lighting conditions. For example, multiple LED light sources can be set to illuminate the object placed in front of the green screen from the top, bottom, front, back and side, so as to capture the visual features of the object under various lighting conditions. These images are captured using a high-resolution camera system to ensure the accuracy and richness of details in the subsequent processing process. Then, in step S2, a depth perception neural network is used to process the captured images. This network is first trained to identify and extract the depth information of the block from each green screen image. The acquisition of depth information uses known camera parameters and optical models to calculate the shape of the object and its actual position in the scene. For example, the network can estimate the distance of the object from the camera and its positioning in three-dimensional space by analyzing the edge and texture changes of the object. In step S3, a convolutional neural network (CNN) is used to further process the image to extract the shadow characteristics of the block. The network combines the depth information and lighting parameters of the object block and is trained to identify the lighting pattern and shadow effect on the surface of the object. For example, for a human model placed under complex lighting, CNN can identify the differences in light occlusion and reflection caused by different parts of the body. In step S4, a virtual environment image is generated based on the green screen environment image matching. This step is performed by image processing personnel to perform basic post-processing to synthesize these virtual elements into different backgrounds, and then semantic segmentation technology is used to identify and separate different blocks in the generated virtual environment image. This technology uses image segmentation algorithms, such as an improved U-Net network, to automatically separate different objects in the virtual environment to facilitate individual material attribute matching. By comparing with a preset material database, each block is assigned corresponding material attributes, such as metallic luster, glass transparency, or diffuse reflection characteristics of cloth. Finally, in step S5, according to the material and lighting parameters of each block, the shadow and reflection area of ​​the block are adjusted and calculated through the back propagation neural network (BP neural network). The algorithm in this step dynamically adjusts its color value according to the light reception and material characteristics of the block surface to simulate the visual effect under natural light. For example, if a block is made of metal and is directly under strong light, its reflective area will be increased and the reflective value range will be adjusted accordingly to enhance its visual highlight effect.

[0043] Furthermore, the object blocks include character blocks and scene blocks.

[0044] It should be noted that for the processing of character blocks, special attention can be paid to capturing the dynamic characteristics of the characters and the skin color, clothing texture and shadow performance under lighting. Use high-resolution photography equipment to capture images of characters under different lighting directions in a green screen environment. These images are then used to train a deep perception neural network, which learns how to extract accurate depth information of the characters and their three-dimensional positions in the scene from the images. In addition, the convolutional neural network analyzes the details of the characters' shadows, such as facial expressions and the folds of clothing, which are crucial for generating high-quality virtual photographic images. For the scenery blocks, the processing process can focus on capturing the geometry of objects, material properties, and interactions in the environment (for example, mutual occlusion between trees and buildings). The image data of the scenery is used to train the same deep perception neural network to determine the precise location and layout of the scenery. The analysis of scenery shadows helps simulate how natural and artificial light effects in the environment affect visual perception, for example, how to correctly render the long shadows of buildings at sunset or the swaying light and shadows of leaves in the breeze.

[0045] Furthermore, the S1 comprises the following steps:

[0046] Set up light sources at different angles and positions in the green screen environment;

[0047] Under the condition of a single light source or a combination of light sources, the green screen environment is photographed from several different angles to obtain an image of the green screen environment;

[0048] Each green screen environment image is annotated, including distance information, light source direction, and light intensity.

[0049] In some embodiments, multiple light sources need to be configured in the green screen environment to ensure that the object can be fully illuminated from all angles. These light sources should include different types (such as spotlights and diffusers) and project light from different heights and angles to simulate various situations of natural light and artificial light. For example, a top spotlight is set to simulate sunlight, while a soft light on the side is used to reduce shadows and enhance details. In this way, various light and shadow effects that may appear on objects in daily environments can be captured. Under the conditions of a single light source or a combination of multiple light sources, the green screen environment is photographed from multiple angles to obtain images of objects under different lighting conditions. This includes shooting from multiple angles such as the front, side, and top to ensure that each angle can capture information about how light affects the surface of the object. The purpose of this operation is to collect enough data to support subsequent neural network training and to ensure that the visual features of the object can be accurately reproduced in complex lighting environments. Each green screen environment image is annotated in detail after capture, which is crucial for subsequent data processing. The annotation content includes information such as the precise distance of the object from the camera, the direction of the light source, and the intensity of the light. These data provide the necessary input for the deep perception neural network to help the network understand and learn the performance of objects under different lighting conditions. For example, for an image, the annotation may indicate that the light source is projected from a 30-degree angle, has an intensity of 800 lumens, and the object is 2 meters away from the camera. These detailed information enable the trained model to more accurately predict and reproduce the lighting effects of objects under similar conditions.

[0050] Further, the S2 comprises the following steps:

[0051] The laser sensing distance and pixel occupancy information of the object in each image are used as the input of the depth perception neural network, and the depth of the object is used as the output of the depth perception neural network for training;

[0052] The depth of the object is obtained through the depth perception neural network, and the relative position of the object in the green screen environment is calculated based on the depth of the object and the shooting angle.

[0053] In some embodiments, key data is first collected from each green screen environment image before performing depth perception neural network training. This includes using a laser sensor to measure the distance of the object from the camera and analyzing the pixel occupancy of the object in the image. The laser sensor provides accurate distance measurement, which is a key parameter for determining the position of an object in three-dimensional space. At the same time, the pixel occupancy information of the object (i.e., the number and distribution of pixels occupied by the object in the image) helps the network understand the size and shape of the object. Together, these data are used as input to the neural network, providing the necessary quantitative information for deep learning. After the data is collected, the depth perception neural network is trained using these detailed measurements. The network is designed to learn how to infer the specific depth of the object from the input laser sensor distance and pixel occupancy information. The training process includes a large number of feedforward and feedback loops, through which the network adjusts its internal parameters (such as weights and biases) to minimize the difference between the predicted depth and the actual measured depth. The goal of network training is to reach an optimization point so that the network can accurately output the depth of the object in any given input image. Once the depth perception neural network is successfully trained and can reliably predict the depth of the object, the next step is to use this depth data to calculate the relative position of the object in the green screen environment. Combining the depth information of the object and the position of the object in the original image, geometric and visual algorithms can be used to calculate the 3D spatial position of the object relative to other objects and the environment. For example, by considering the shooting angle and the depth of the object, the exact position where the object should appear in the virtual scene can be determined, which is crucial for the subsequent image synthesis and the correct rendering of light and shadow effects.

[0054] Furthermore, the method of extracting the shadow features of the blocks based on the convolutional neural network includes annotating the shadows of the blocks according to the three-point-one-line principle to divide the shadows of individual blocks, and outputting the shadow pixel matrix of the blocks through the convolutional neural network for the annotated blocks.

[0055] It should be noted that before starting to use the convolutional neural network (CNN) for processing, the shadow features of the object need to be accurately annotated. This step is carried out using the "three-point one-line principle", that is, the position and shape of the shadow are calibrated by determining the linear relationship between the light source, the object and the shadow. For example, if a cylinder is placed directly under the top light source, its shadow will extend directly on the surface below the cylinder. Based on the shape of the object and the direction of the light source, the annotator will use image editing tools to manually or semi-automatically mark the outline of the shadow in the green screen environment image, including the length, width and direction of the shadow. The annotated image is processed using a convolutional neural network to extract the shadow features of the block and convert them into a pixel matrix format. CNN is trained to recognize and analyze the shadows of objects under various lighting and geometric conditions. The network structure design includes multiple convolutional layers, activation layers, and pooling layers, which can effectively capture and enhance the features of the shadow area, such as boundary clarity and grayscale level. The convolution operation of each layer gradually constructs a global shadow representation by extracting local features, and finally outputs the pixel matrix of the shadow. The result of CNN processing is a detailed pixel matrix of the shadow of the object, which describes in detail the color value and brightness of each pixel of the shadow. This data structure is stored in a format that can be directly applied to image synthesis, ensuring that the shadow of the object in the virtual environment accurately matches the actual lighting conditions. For example, by analyzing the shadow pixel matrix, the shadow density and color of the corresponding object in the virtual environment can be adjusted so that it can be visually integrated with other environmental elements.

[0056] Furthermore, the lighting parameters include the relative position of the lighting in the green screen environment, the lighting intensity and the lighting angle.

[0057] It should be noted that the relative position of the light source in the green screen environment has a direct impact on how the object is illuminated and the shadows it produces. For example, a top light source will produce a shadow at the bottom of the object, while a side light source may produce a longer shadow on the opposite side. In virtual photography, by precisely controlling the position of the light source, it is possible to simulate daylight, artificial light, or any lighting effect required for a specific scene, thereby enhancing the three-dimensionality and depth of the image. The intensity of the light determines the brightness of the image and its revelation of the surface details of the object. High light intensity can reduce the shadow area and make the details of the object more clearly visible, but too strong light may cause overexposure and loss of details. On the contrary, low light intensity enhances the contrast of the image, but may cause some details to fall into shadows. By adjusting the light intensity, the visual performance of the object can be optimized to ensure that the image maintains good visual effects on different viewing devices. The angle of light is another key parameter that determines the direction and length of the shadow. Changes in the angle of light directly affect the direction and length of the shadow projection, which is used in visual art to enhance the realism of the scene or create specific visual effects. For example, using a low-angle light source can simulate the long shadows of sunset, while a high-angle light source can simulate the short shadows of midday.

[0058] Furthermore, the performing semantic segmentation based on the virtual environment and performing material matching for objects in the virtual environment image according to the semantic segmentation result includes the following steps:

[0059] The green screen environment image is processed through the semantic segmentation algorithm to automatically identify and segment different objects and background elements in the image;

[0060] According to the segmentation result, the material of the object is obtained;

[0061] Block-based materials traverse the material table and match corresponding material properties, including surface texture, glossiness, and color.

[0062] In some embodiments, first, a semantic segmentation algorithm is used to process the image under the green screen background. The semantic segmentation algorithm, which can be a Fully Convolutional Network (FCN) or SegNet based on deep learning, can accurately identify and segment each object and background in the image. These algorithms train the model to recognize the contours and features of different objects by learning a large amount of image data. In the application of virtual photography, this means that elements such as people, furniture, and vegetation in the image can be automatically distinguished and clearly separated from the background. Once the semantic segmentation of the objects is completed, the next step is to analyze and determine the material properties of each independent object. This includes judging the material by analyzing the reflectivity, texture details, and color distribution of the surface of the object. For example, a smooth and highly reflective surface may indicate that the object is made of metal, while a rough and light-absorbing surface may be wood or stone. This recognition process utilizes color and texture analysis tools in image processing technology, such as gray level co-occurrence matrix (GLCM) or color histogram analysis, as well as calculation of surface reflectance characteristics. Finally, based on the identified material properties, the system will traverse the predefined material database to find matching material properties for each object. This database contains detailed descriptions of various materials, including their surface texture, glossiness, color, and lighting response characteristics. Through precise material matching, each object can be assigned the material properties that best suit its visual performance, which are then used to adjust and optimize the visual rendering of objects in the virtual environment. For example, an object identified as metal will have its highlights increased when rendered, while a wooden object will show its natural texture and warm tones.

[0063] Furthermore, the step of fusing the shadows of several objects in the virtual environment image and calculating the reflective area of ​​the objects comprises the following steps:

[0064] Based on the positional relationship, shadow characteristics, and relative height and angle between each object, identify the combination of objects that may cause light occlusion;

[0065] The overlapped images of the identified occluded objects are subjected to weight reduction fusion to generate the final image;

[0066] For the identified occluding blocks, the light propagation model is applied to simulate the path of light from the light source to the surface of each block and determine the illuminated area of ​​each affected block.

[0067] In some embodiments, it is first necessary to analyze the spatial position relationship, shadow characteristics, and relative height and angle of each object in the virtual environment. This analysis is based on the three-dimensional model data of the object and its placement information in the virtual environment. Through image processing and scene analysis techniques, such as deep learning algorithms analyzing forward and backward vector learning, the system can identify which objects affect each other's lighting performance due to mutual occlusion. For example, a tall building may cast a shadow on pedestrians on the street under the sun at a specific time, or the crown of a tree may block the vegetation below. After identifying the occlusion relationship, the shadows of these objects are then fused. This includes "weight reduction fusion" of overlapping shadow areas, that is, using algorithms to reduce or adjust the shadow density of the overlapping areas to reflect the possible light attenuation in natural light environments. This step uses advanced image processing techniques, such as mask stacking and pixel-level shadow adjustment, to ensure the natural transition and visual consistency of shadows. Finally, in order to accurately simulate the propagation path of light from the light source to the surface of the object, the system uses a light propagation model. The model takes into account the type of light source (such as sunlight, artificial light, etc.), environmental conditions (such as the scattering effect of media such as fog and smoke), and the reflection and absorption characteristics of objects. By calculating the model, the illuminated area of ​​each affected object can be accurately determined, including those areas that are partially blocked by other objects. These calculations help optimize the lighting effects on the surfaces of objects, making them more visually realistic and reasonable. For example, if a car is partially blocked by a building, the model can not only calculate the light intensity of the unblocked part of the car, but also estimate the scattered light effect of the blocked part, thereby adjusting the reflection and shadow effects of the car body to match the lighting changes in actual observations.

[0068] Furthermore, the controlling the color value of the object in the virtual environment image by the reflective area and the reflective value interval includes:

[0069] Applying a dynamic color value adjustment algorithm to adjust the color value of objects in the virtual environment image according to the reflective area and the selected reflective value interval;

[0070] The adjusted color values ​​of the objects are post-processed, including color balancing, contrast optimization, and brightness and darkness adjustment.

[0071] In some embodiments, a dynamic color value adjustment algorithm is used to adjust the color value of the object according to the reflective area of ​​the object and the pre-set reflective value interval. The key components of the dynamic color value adjustment algorithm include: 1. Reflective area calculation: The reflective area refers to the actual area of ​​the object surface that receives light, taking into account the geometric shape of the object, the surface characteristics, and the angle and intensity of the light. The formula for calculating the reflective area can be based on a physical optics model, such as using the Fresnel equation to estimate the effect of the interaction between light and the surface of an object. These calculation results will determine the intensity of light on a specific area of ​​the object surface. 2. Reflective value interval setting: A reflective value interval is pre-set based on the material properties of the object, such as gloss, transparency, and color. For example, a metal surface may have a high reflective value interval, while wood or stone is set in a lower reflective value interval. This interval defines the range of color value changes of the object when it receives maximum and minimum light. 3. Color value mapping and adjustment: Color value adjustment is implemented through a mapping algorithm that converts the reflective area and reflective value interval into specific color adjustment parameters. The mapping function is usually nonlinear to simulate the complex changes in the color of objects under natural light. The mapping result will directly affect the brightness, contrast and saturation of the object.

[0072] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for generating virtual photographic images based on a neural network, characterized in that: The following steps are involved: S1. Build a green screen environment and shoot green screen environment images under different lighting directions; S2, based on a plurality of green screen environment images, extracting block depth information from the green screen environment images through a depth perception neural network, and calculating the relative position of the object block in the green screen environment; S3, extracting the shadow of the block based on the convolutional neural network, taking the depth information of the block, the relative position of the block in the green screen environment and the illumination parameters as the input elements of the BP neural network, and taking the shadow of the block as the output element to train the BP neural network; S4, generating a virtual environment image based on the green screen environment image matching, performing semantic segmentation based on the virtual environment, and performing material matching for objects in the virtual environment image according to the semantic segmentation result; S5, setting the illumination parameters and selecting the reflective value interval according to the material of the objects in the virtual environment, calculating the shadow of the objects based on the trained BP neural network, fusing the shadows of several objects in the virtual environment image, and calculating the reflective area of ​​the objects, regulating the color value of the objects in the virtual environment image by the reflective area and the reflective value interval, and generating a virtual photographic image; The step of fusing the shadows of several objects in the virtual environment image and calculating the reflective areas of the objects comprises the following steps: Based on the positional relationship, shadow characteristics, and relative height and angle between each object, identify the combination of objects that may cause light occlusion; The overlapped images of the identified occluded objects are subjected to weight reduction fusion to generate the final image; For the identified occluding blocks, a light propagation model is applied to simulate the path of light from the light source to the surface of each block, and the light receiving area of ​​each affected block is determined. The light receiving area is the reflective area.

2. The method for generating virtual photographic images based on a neural network according to claim 1, characterized in that: The object blocks include character blocks and scene blocks.

3. The method for generating virtual photographic images based on a neural network according to claim 1, characterized in that: The S1 comprises the following steps: Set up light sources at different angles and positions in the green screen environment; Under the condition of a single light source or a combination of light sources, the green screen environment is photographed from several different angles to obtain an image of the green screen environment; Each green screen environment image is annotated, including distance information, light source direction, and light intensity.

4. The method for generating virtual photographic images based on a neural network according to claim 3, characterized in that: The S2 comprises the following steps: The laser sensing distance and pixel occupancy information of the object in each image are used as the input of the depth perception neural network, and the depth of the object is used as the output of the depth perception neural network for training; The depth of the object is obtained through the depth perception neural network, and the relative position of the object in the green screen environment is calculated based on the depth of the object and the shooting angle.

5. The method for generating virtual photographic images based on a neural network according to claim 1, characterized in that: The method of extracting shadow features of blocks based on convolutional neural network includes marking the shadows of blocks according to the three-point-one-line principle to divide the shadows of individual blocks, and outputting the shadow pixel matrix of the marked blocks through the convolutional neural network; the three-point-one-line principle includes calibrating the position and shape of the shadow by determining the linear relationship between the light source, the block and the shadow.

6. The method for generating virtual photographic images based on a neural network according to claim 1, characterized in that: The lighting parameters include the relative position of the lighting in the green screen environment, the lighting intensity and the lighting angle.

7. The method for generating virtual photographic images based on a neural network according to claim 5, characterized in that: The method of performing semantic segmentation based on the virtual environment and performing material matching for objects in the virtual environment image according to the semantic segmentation result comprises the following steps: The green screen environment image is processed through the semantic segmentation algorithm to automatically identify and segment different objects and background elements in the image; According to the segmentation result, the material of the object is obtained; Block-based materials traverse the material table and match corresponding material properties, including surface texture, glossiness, and color.

8. The method for generating virtual photographic images based on a neural network according to claim 1, characterized in that: The method of regulating the color value of the object in the virtual environment image by the reflective area and the reflective value interval includes: Applying a dynamic color value adjustment algorithm to adjust the color value of objects in the virtual environment image according to the reflective area and the selected reflective value interval; The adjusted color values ​​of the objects are post-processed, including color balancing, contrast optimization, and brightness and darkness adjustment.

Images