Image rendering method, device, equipment and chip

Abstract

The invention provides an image rendering method and device, equipment and a chip, and the method comprises the steps: obtaining a to-be-rendered target image in response to a user operation, and carrying out the fusion of the depth information and normal information of the target image, and obtaining the fusion information; performing feature extraction on the fusion information to obtain ambient light shielding information of the target image; according to the ambient light shielding information, correcting the illumination information of the target image to obtain corrected illumination information; and performing illumination rendering on the target image according to the corrected illumination information. Therefore, the ambient light shielding information of the target image is determined by using the depth information and the normal information of the target image at the terminal, and is applied to the illumination information of the target image, so that the rendering image quality can be improved and the rendering power consumption can be reduced while the real-time performance is ensured.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to an image rendering method, device, equipment and chip. BACKGROUND

[0002] With the development of computer graphics technology, players have higher and higher requirements for the realism and visual effects of game pictures, and expect to experience exquisite picture quality close to desktop-level games on mobile terminals.

[0003] However, due to the limited resources such as GPU (Graphics Processing Unit) computing power and memory of mobile terminals, it is difficult to ensure real-time smooth running of game pictures while taking into account high-quality rendering effects. SUMMARY

[0004] The present disclosure aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, the present disclosure provides an image rendering method, device, equipment and chip, which determines ambient light shading information of a target image by using depth information and normal information of the target image on a terminal, and applies the ambient light shading information to lighting information of the target image, so as to improve rendering picture quality while ensuring real-time performance and reducing rendering power consumption.

[0006] The first aspect embodiment of the present disclosure provides an image rendering method, which comprises:

[0007] In response to a user operation, a target image to be rendered is obtained, and depth information and normal information of the target image are fused to obtain fusion information;

[0008] Feature extraction is performed on the fusion information to obtain ambient light shading information of the target image;

[0009] According to the ambient light shading information, lighting information of the target image is corrected to obtain corrected lighting information;

[0010] According to the corrected lighting information, the target image is rendered with lighting.

[0011] The second aspect embodiment of the present disclosure provides an image rendering device, which comprises:

[0012] A processing module is configured to, in response to a user operation, obtain a target image to be rendered, and fuse depth information and normal information of the target image to obtain fusion information;

[0013] A feature extraction module is configured to perform feature extraction on the fusion information to obtain ambient light shading information of the target image;

[0014] The correction module is used to correct the illumination information of the target image based on the ambient light occlusion information to obtain the corrected illumination information.

[0015] The rendering module is used to perform lighting rendering on the target image based on the corrected lighting information.

[0016] A third aspect of this disclosure provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the image rendering method as described in the first aspect above.

[0017] A fourth aspect of this disclosure provides a chip including an interface circuit and a processing circuit coupled to each other, the interface circuit being used to input or output signals, and the processing circuit being configured to perform the image rendering method as described in the first aspect above.

[0018] The fifth aspect of this disclosure provides a non-transitory computer-readable storage medium having stored computer program instructions thereon, which, when executed by a processor, implement the image rendering method as described in the first aspect above.

[0019] A sixth aspect of this disclosure provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the image rendering method as described in the first aspect above.

[0020] The image rendering method, apparatus, device, and chip disclosed herein respond to user operations by acquiring a target image to be rendered, fusing the depth and normal information of the target image to obtain fused information; extracting features from the fused information to obtain ambient occlusion information of the target image; correcting the lighting information of the target image based on the ambient occlusion information to obtain corrected lighting information; and rendering the target image based on the corrected lighting information. Therefore, by utilizing the depth and normal information of the target image at the terminal to determine the ambient occlusion information of the target image and applying it to the lighting information of the target image, rendering quality can be improved while ensuring real-time performance and reducing rendering power consumption.

[0021] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0023] Figure 1 This is a schematic flowchart of an image rendering method provided in an embodiment of the present disclosure;

[0024] Figure 2 This is a schematic flowchart of another image rendering method provided in an embodiment of the present disclosure;

[0025] Figure 3 This is a schematic diagram of a neural network structure provided in an embodiment of the present disclosure;

[0026] Figure 4 This is a schematic flowchart of another image rendering method provided in an embodiment of the present disclosure;

[0027] Figure 5 This is a schematic flowchart illustrating an image rendering process provided in an embodiment of the present disclosure;

[0028] Figure 6 This is a schematic diagram of the structure of an image rendering apparatus provided in an embodiment of the present disclosure;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0030] Figure 8 This is a schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. Detailed Implementation

[0031] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0032] With the development of computer graphics technology, players have higher and higher requirements for the realism of game graphics. The core element to improve the realism of game graphics is the lighting effect. If the lighting effect can be simulated very closely with real life, then the image will present a very high sense of realism and visual credibility.

[0033] Early games could only achieve direct lighting effects, meaning that light emitted from a light source, shone on an object, and reflected into the player's eyes; the lighting calculation was then complete. However, with technological advancements, to achieve more realistic game graphics, it's necessary to calculate not only direct lighting but also indirect lighting. This means that after light emitted from a light source and shone on an object, in addition to calculating the light reflected from the object itself to the player's eyes, it's also necessary to calculate the multiple reflections, refractions, and absorptions of light by the object, as well as the changes in light and shadow caused by reflections and refractions between objects. Through comprehensive calculations, the final effect of light and shadow is displayed (reflected) to the player's eyes, thus achieving the effect of global illumination.

[0034] Global illumination works well for static graphics because static graphics can wait for rendering time. However, for games, continuous dynamic scenes must have sufficient rendering speed to ensure that the frames per second (FPS) is met. Therefore, the application of global illumination must ensure corresponding running efficiency while improving the realism of the scene; otherwise, it will break the balance between the scene effect and the frame rate.

[0035] However, mobile devices, due to limited GPU computing power and memory resources, struggle to ensure smooth real-time gameplay while maintaining high-quality rendering effects.

[0036] To address the aforementioned issues, this disclosure proposes an image rendering method, apparatus, device, and chip. By utilizing the depth and normal information of the target image at the terminal, the ambient light occlusion information of the target image is determined and applied to the lighting information of the target image. This achieves improved rendering quality and reduced rendering power consumption while ensuring real-time performance.

[0037] The image rendering method, apparatus, device, and chip of this disclosure are described below with reference to the accompanying drawings.

[0038] Figure 1 This is a schematic flowchart of an image rendering method provided in an embodiment of the present disclosure.

[0039] It should be noted that the image rendering method of this disclosure can be applied to an image rendering apparatus. In some possible embodiments, the image rendering apparatus can be configured in an electronic device or chip so that the electronic device or chip can perform image rendering functions. Additionally, in some possible embodiments, the image rendering apparatus can also be software within an electronic device.

[0040] In any embodiment of this disclosure, the chip can be integrated into an electronic device. The chip includes a Central Processing Unit (CPU), an Image Signal Processing (ISP), an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a System-on-A-Chip (SOC), a Reduced Instruction Set Computer (RISC), etc., which will not be listed here.

[0041] Among them, electronic devices include, but are not limited to: terminals.

[0042] In this context, a terminal is a user-side entity used to receive or transmit signals, such as a mobile phone. A terminal can also be called a terminal device (UE), user equipment (UE), mobile station (MS), mobile terminal device (MT), etc. Terminals can be mobile phones with communication functions, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminals, augmented reality (AR) terminals, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments disclosed herein do not limit the specific technologies or device forms used in the terminals.

[0043] like Figure 1 As shown, the image rendering method may include the following steps S101 to S102:

[0044] In step S101, in response to user operation, the target image to be rendered is acquired, and the depth information and normal information of the target image are fused to obtain fused information.

[0045] User actions can be various interactive behaviors within a graphical user interface. For example, a user dragging the mouse to rotate the viewpoint, or a player moving their character to change their viewing position.

[0046] The target image to be rendered can be an image determined in response to user actions. For example, after the user rotates the viewpoint, it is necessary to determine the content of the image to be rendered next, which may involve changes in the scene's viewpoint, adjustments to object positions, etc. The scene seen from the new viewpoint is the target image to be rendered.

[0047] The depth information of the target image is used to indicate the distance of each pixel in the target image to the observation point (such as an image acquisition device or a reference plane).

[0048] Depth information can be used to determine the position (distance) of objects in space and their occlusion relationships, and to reconstruct the three-dimensional (3D) spatial structure of the scene in the target image.

[0049] Among them, the normal information of the target image is used to indicate the "orientation" of each pixel on the surface of the object in the target image (which can be represented by a "normal vector, which is a vector perpendicular to the surface of the object).

[0050] By using normal information, we can determine how an object's surface reflects light (affecting brightness and shadow effects) and determine the direction of occlusion. For example, surfaces with normals pointing upwards (such as the ground) are more likely to receive ambient light, while surfaces with normals pointing inwards (such as corners) are more likely to be occluded.

[0051] In one implementation of this disclosure, the depth information and normal information of the target image can be concatenated to obtain fused information. For example, the depth information of the target image is represented by a depth map, and the normal information of the target image is represented by a normal map. The depth map is a single-channel feature map (each pixel is a depth value), and the normal map is a 3-channel feature map (each pixel is a three-dimensional normal vector, which can be divided into three channels: x, y, and z). The depth map and the normal map can be concatenated according to the channel dimension to obtain a 4-channel fused feature map, which is the aforementioned fused information.

[0052] Step S102: Extract features from the fused information to obtain ambient light occlusion information of the target image.

[0053] Ambient occlusion (AO) describes the effect of objects blocking diffuse light when they intersect or are close together. It can solve or improve problems such as light leaks, blurry object edges ("floating"), and unrealistic shadows. In terms of scene detail, AO can solve or improve the unclear representation of gaps, wrinkles, corners, and small objects in a scene. It comprehensively improves details, especially shadows in dark areas, enhances the spatial depth and realism of the image, and strengthens and improves the contrast between light and dark, thus enhancing the artistic quality of the image.

[0054] In some embodiments, a neural network running inside the terminal GPU can be used to extract features from the fused information to obtain ambient light occlusion information of the target image.

[0055] Step S103: Based on the ambient light occlusion information, the illumination information of the target image is corrected to obtain the corrected illumination information.

[0056] The illumination information of the target image includes direct illumination information (such as sunlight, lamplight, etc.) and indirect illumination information (such as ambient light, reflected light, etc.).

[0057] Since rendering a target image based directly on its lighting information may result in some areas of the target image being too bright or too dim, making the final rendered target image appear unrealistic, in some embodiments, the lighting information of the target image can be corrected based on the ambient light occlusion information of the target image to make the corrected lighting information more consistent with physical reality.

[0058] Step S104: Perform lighting rendering on the target image based on the corrected lighting information.

[0059] In some embodiments, rendering the target image based on the corrected lighting information can make the changes in brightness and darkness on the surface of the object in the rendered image more natural and the light and shadow layers richer.

[0060] The image rendering method of this disclosure, in response to a user operation, acquires a target image to be rendered, and fuses the depth information and normal information of the target image to obtain fused information; extracts features from the fused information to obtain ambient light occlusion information of the target image; corrects the lighting information of the target image based on the ambient light occlusion information to obtain corrected lighting information; and renders the target image based on the corrected lighting information. Therefore, by using the depth information and normal information of the target image at the terminal to determine the ambient light occlusion information of the target image and applying it to the lighting information of the target image, rendering quality can be improved while ensuring real-time performance and reducing rendering power consumption.

[0061] This disclosure provides another image rendering method. Figure 2 This is a schematic flowchart of another image rendering method provided in an embodiment of the present disclosure.

[0062] like Figure 2 As shown, the image rendering method may include the following steps S201 to S204:

[0063] In step S201, in response to user operation, the target image to be rendered is acquired, and the depth information and normal information of the target image are fused to obtain fused information.

[0064] It should be noted that the explanation of this step can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0065] Step S202: Downsample the fused information to obtain the target fused information.

[0066] In some embodiments, the fused information can be downsampled by different sampling factors to obtain the corresponding target fused information. For example, the fused information can be downsampled by 2... n This multiplies the value of the target fusion information.

[0067] Therefore, by downsampling the fused information, we can reduce the data dimensionality while preserving the key structure, thereby reducing the amount of subsequent computation and improving processing efficiency.

[0068] Step S203: Extract features from the target fusion information to obtain feature information, and filter the target fusion information to obtain key information.

[0069] In this embodiment of the disclosure, feature extraction of the target fusion information can capture the deep-seated characteristics of the target fusion information, and information filtering of the target fusion information can accurately screen out key information related to ambient light occlusion.

[0070] In some embodiments, neural networks can be used to extract features and filter information from target fusion information.

[0071] In one implementation of this disclosure, the target fusion information corresponding to the downsampling factor is input into the corresponding encoder in the neural network for feature extraction to obtain the feature information output by the corresponding encoder; wherein, at least two encoders in the neural network correspond to different downsampling factors.

[0072] Therefore, by downsampling at different multiples (such as 2, 4, 8 times downsampling), the neural network can capture the features of the target fusion information at different scales. This multi-scale feature extraction enables the neural network to focus on both local and global information at the same time, improving the neural network's ability to perceive and understand complex fusion information.

[0073] In one implementation of this disclosure, target fusion information downsampled by a fixed factor is input into the bottleneck layer of a neural network for information filtering to obtain key information.

[0074] The bottleneck layer is a structure introduced into neural networks to compress information and extract key features. It is commonly found in the intermediate layers of Convolutional Neural Networks (CNNs), such as the Bottleneck structure in ResNet.

[0075] Bottleneck layers typically consist of three convolutional layers: a 1×1 convolutional layer, used to reduce the number of input channels (dimensionality) to reduce the computational cost of subsequent convolutional layers; a 3×3 convolutional layer, used to perform convolution operations on the reduced-dimensional feature map to extract features; and a 1×1 convolutional layer, used to restore the number of channels to the original dimension for use by the next layer.

[0076] Therefore, the redundant dimensions of the input data can be directly reduced by lowering the spatial resolution (e.g., by 4 times the sampling), and the feature channels can be further compressed by operations such as 1x1 convolution in the bottleneck layer (e.g., from 256 dimensions to 64 dimensions) to remove redundant features and retain key feature information.

[0077] Step S204: Determine ambient light occlusion information based on feature information and key information.

[0078] In some embodiments, after using a neural network to extract features and filter information from the target fusion information to obtain feature information and key information, the neural network can be used to determine ambient light occlusion information based on the feature information and key information.

[0079] In one implementation of this disclosure, the neural network further includes a multi-layer decoder, with a correspondence between the decoder and the encoder. Thus, for the bottom-level target decoder, key information and feature information output by the encoder corresponding to the target decoder are input to the target decoder to obtain candidate ambient light occlusion information output by the target decoder. For any decoder other than the target decoder, candidate ambient light occlusion information output by the previous layer decoder and feature information output by the encoder corresponding to the decoder are input to the decoder to obtain candidate ambient light occlusion information output by the decoder. The candidate ambient light occlusion information output by the top-level decoder is determined as ambient light occlusion information.

[0080] Therefore, for the bottom-level target decoder, by fusing key information with the feature information output by the corresponding encoder, the key features of ambient light occlusion can be accurately captured. For other decoders, by combining the candidate ambient light occlusion information output by the previous layer decoder with the feature information output by its own corresponding encoder, the information is progressively advanced and optimized, enabling each layer decoder to process based on richer contextual information. Finally, the candidate ambient light occlusion information output by the top-level decoder is determined as ambient light occlusion information, improving the accuracy and completeness of ambient light occlusion information generation and better simulating the real ambient light occlusion effect.

[0081] In one implementation of this disclosure, the candidate ambient light occlusion information output by the target decoder is obtained by adjusting the ambient light occlusion information generated by the target decoder based on key information. The target decoder generates ambient light occlusion information based on the feature information output by the encoder corresponding to the target decoder. The candidate ambient light occlusion information output by the decoder is obtained by adjusting the ambient light occlusion information generated by the decoder based on the candidate ambient light occlusion information output by the previous layer decoder. The decoder generates ambient light occlusion information based on the feature information output by the encoder corresponding to the decoder.

[0082] Thus, the target decoder uses key information to adjust the candidate ambient light occlusion information generated based on the feature information output by the corresponding encoder, making the candidate ambient light occlusion information generated by the target decoder closer to physical reality; other decoders use the output of the previous layer as a reference to adjust the candidate ambient light occlusion information generated based on the feature information output by their own corresponding encoder, realizing the effective transfer and gradual optimization of information between different layers, and progressively improving the accuracy of information.

[0083] In some embodiments, the neural network may adopt a network structure in the form of UNet.

[0084] As an example, such as Figure 3 As shown, the neural network includes 3 encoders, 3 decoders, and 1 bottleneck layer.

[0085] For each encoder n, the input is the fused information downsampled by 2. n The output is the corresponding feature information, multiplied by a factor of 1.

[0086] For the bottleneck layer, the input is a fixed downsampling factor of the fused information (such as 8 times, 16 times, etc.), and the output is key information;

[0087] For the lowest-level target decoder (decoder 2), the inputs are the output of the corresponding encoder and the output of the bottleneck layer;

[0088] For each decoder except the bottom-level target decoder, the input is the output of the corresponding encoder and the output of the previous layer decoder;

[0089] For each decoder n, the output is the corresponding candidate ambient occlusion information.

[0090] Figure 3 In the above, for encoder 0, the input is the fused information downsampled by 1, which is the original fused information; for encoder 1, the input is the fused information downsampled by 2; and for encoder 2, the input is the fused information downsampled by 4.

[0091] Among them, the number of feature channels in the input and output of the neural network is as follows: Figure 3 As shown, for encoders 0, 1, 2, and the bottleneck layer, the number of input feature channels is 4, but the number of output feature channels differs. Specifically, encoders 0 and the bottleneck layer both output 4 feature channels, but encoders 1 and 2 both output 8 feature channels. For decoders 0, 1, and 2, the number of output feature channels is 1, but the number of input feature channels differs. For decoder 0, the number of input feature channels is 8+4; for decoder 1, it is 8+1; and for decoder 2, it is 4+1.

[0092] Each encoder and decoder consists of two convolutional layers connected by a ReLU (Rectified Linear Unit) layer.

[0093] Among them, the candidate ambient light occlusion information output by the top-level decoder can be determined as the ambient light occlusion information of the target image, that is, the candidate ambient light occlusion information output by decoder 1 can be determined as the ambient light occlusion information of the target image.

[0094] In the above process, feature extraction of the fused information is performed through a neural network to obtain the ambient light occlusion information of the target image, which can effectively reduce the amount of computation. For example, for a four-channel image with a resolution of 1024*1024, the amount of computation is 821.89M and the number of parameters is 5.21K.

[0095] Step S205: Based on the ambient light occlusion information, the illumination information of the target image is corrected to obtain the corrected illumination information.

[0096] Step S206: Perform lighting rendering on the target image based on the corrected lighting information.

[0097] It should be noted that the explanations of steps S205 and S206 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0098] The image rendering method of this disclosure, in response to a user operation, acquires a target image to be rendered, and fuses the depth information and normal information of the target image to obtain fused information; downsamples the fused information to obtain target fused information; extracts features from the target fused information to obtain feature information, and filters the target fused information to obtain key information; determines ambient light occlusion information based on the feature information and key information; corrects the lighting information of the target image based on the ambient light occlusion information to obtain corrected lighting information; and renders the target image based on the corrected lighting information. Thus, feature extraction can capture the deep-level characteristics of the target's geometric information, while information filtering can accurately filter out key information related to ambient light occlusion. Determining ambient light occlusion information based on feature information and key information can more accurately reflect the light and shadow occlusion situation of the target image.

[0099] This disclosure provides another image rendering method. Figure 4 This is a schematic flowchart of another image rendering method provided in an embodiment of the present disclosure.

[0100] like Figure 4 As shown, the image rendering method may include the following steps S401 to S405:

[0101] In step S401, in response to the user operation, the target image to be rendered is acquired, and the depth information and normal information of the target image are fused to obtain fused information.

[0102] Step S402: Extract features from the fused information to obtain ambient light occlusion information of the target image.

[0103] It should be noted that the explanations of steps S401 and S402 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0104] Step S403: Obtain the illumination information of the target image, wherein the illumination information includes ambient light information, diffuse reflection light information and specular reflection light information.

[0105] In some embodiments, ambient light information, diffuse light information, and specular light information of the target image can be obtained from the lighting model. For example, ambient light information, diffuse light information, and specular light information of the target image can be obtained from the Bling-Phong lighting model.

[0106] Step S404: Based on ambient light occlusion information, ambient light information, diffuse reflection light information, and specular reflection light information, determine the corrected illumination information.

[0107] In one implementation of this disclosure, ambient light occlusion information and ambient light information are fused to obtain target ambient light information; target ambient light information, diffuse reflection light information and specular reflection light information are superimposed to obtain corrected illumination information.

[0108] Therefore, by fusing ambient light occlusion information with ambient light information to generate target ambient light, the local shadow details caused by geometric occlusion on the object surface can be accurately simulated, enhancing the spatial layering of ambient light. Furthermore, by superimposing target ambient light with diffuse and specular light, the ambient light, material diffuse reflection characteristics, and specular reflection effects can be unified and integrated to improve the realism of the image.

[0109] In another implementation of this disclosure, the product of ambient light occlusion information and ambient light information is determined as the corrected ambient light information; the sum of the corrected ambient light information, diffuse reflection light information, and specular reflection light information is determined as the corrected illumination information.

[0110] Therefore, by introducing nonlinear relationships through multiplication operations, the illumination effect of ambient light on the surface of an object can be accurately simulated; by adding operations, the organic fusion of different light components can be achieved, simulating lighting effects close to those in the real world.

[0111] Step S405: Perform lighting rendering on the target image based on the corrected lighting information.

[0112] It should be noted that the explanation of this step can be found in the relevant description in any embodiment of this disclosure, and will not be repeated here.

[0113] The image rendering method of this disclosure, in response to a user operation, acquires a target image to be rendered, and fuses the depth information and normal information of the target image to obtain fused information; extracts features from the fused information to obtain ambient light occlusion information of the target image; acquires the lighting information of the target image, wherein the lighting information includes ambient light information, diffuse light information, and specular light information; determines corrected lighting information based on the ambient light occlusion information, ambient light information, diffuse light information, and specular light information; and renders the target image according to the corrected lighting information. Thus, by acquiring the lighting information of the target image containing ambient light, diffuse light, and specular light, and fusing it with ambient light occlusion information for correction, the corrected lighting information more realistically reflects the lighting conditions of objects in the target image.

[0114] The following examples illustrate this. Figure 5 This is a schematic flowchart illustrating an image rendering process provided in an embodiment of the present disclosure.

[0115] The application scenario of this disclosure is real-time game rendering on mobile devices. This disclosure uses geometric buffers such as depth and normal (GBuffer) to generate ambient occlusion information (such as AO Map), and applies it to global illumination to simulate the diffuse occlusion of natural ambient light, enhance the shadow effect of details such as corners and edges of the model surface (such as corners of walls and gaps in furniture), and improve the realism of the scene.

[0116] like Figure 5 As shown, the process includes:

[0117] 1. Obtain GBuffer.

[0118] GBuffer is a commonly used data structure in real-time rendering (especially deferred rendering techniques) to store the geometric information and material properties (such as color, depth, normals, roughness, etc.) of each pixel in the scene during the rendering process. This information is stored in multiple textures (referred to as GBuffer textures), and subsequent steps such as lighting calculations can directly read data from the GBuffer, thereby efficiently achieving complex lighting effects (such as multi-source lighting).

[0119] like Figure 5 As shown in this embodiment of the disclosure, for game screen images (i.e. target images to be rendered) rendered using deferred rendering, the GBuffer can be obtained after vertex shading and rasterization.

[0120] In this embodiment of the disclosure, the depth information and normal information of the aforementioned game screen image can be obtained.

[0121] 2. Generate AO Map.

[0122] like Figure 5 As shown in this embodiment of the disclosure, an AO Map can be generated based on the obtained GBuffer.

[0123] In one implementation of this disclosure, 1-channel Depth information and 3-channel Normal information can be input into a neural network to output a 1-channel AO Map.

[0124] As an example, suppose the depth information in the GBuffer is D∈R H*W*1 The normal information is N∈R H*W*3 The neural network used to calculate the AO Map is f. AO (·), then we have:

[0125] AO = f AO (D,N),

[0126] Where, AO∈R H*W*1 .

[0127] Where D represents depth information, such as a depth map; N represents normal information, such as a normal map; AO represents ambient occlusion information, such as an ambient occlusion map; R represents the real number field, for example, indicating that the depth value of each pixel in the depth map is a real number (e.g., 1.5 meters, 3.2 meters); H*W*1 and H*W*3 represent the tensors of the corresponding information, corresponding to the three dimensions of "height × width × number of channels", where H represents height and W represents width, for example, D∈R H*W*1 This represents a single-channel depth map with a resolution of "height H rows × width W columns", where N ∈ R. H*W*3 This represents a 3-channel normal map with a resolution of "height H rows × width W columns", AO∈R. H*W*1 This represents a single-channel ambient occlusion map with a resolution of "height H rows × width W columns".

[0128] The neural network used to calculate the AO Map can be, for example, as follows: Figure 3 The network structure shown is in the form of UNet.

[0129] It should be noted that in this embodiment of the present disclosure, the 1-channel Depth information and the 3-channel Normal information need to be concatenated to obtain fused information. After downsampling the fused information by different sampling multiples, it is input into different layers of the neural network. The explanation of this process can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0130] III. Obtaining illumination information.

[0131] In this embodiment of the disclosure, the lighting information of the aforementioned game screen image can be obtained. The lighting information may include ambient light information, diffuse light information, and specular light information of the aforementioned game screen image.

[0132] In one implementation of this disclosure, ambient light information, diffuse light information, and specular light information of the aforementioned game screen image can be obtained from a lighting model. For example, ambient light information, diffuse light information, and specular light information of the target image can be obtained from a Bling-Phong lighting model.

[0133] It should be noted that the lighting information in this step is also calculated based on the GBuffer obtained in step one.

[0134] IV. Correct lighting information.

[0135] like Figure 5As shown in this embodiment, the acquired illumination information can be corrected based on the AO Map.

[0136] As an example, applying AO Map to the rendering pipeline weights the acquired lighting information by occlusion, making the occluded pixels darker and improving the sense of depth and realism.

[0137] In one implementation of this disclosure, the illumination information includes ambient light information, diffuse reflection light information, and specular reflection light information. The corrected illumination information can be determined based on the ambient light occlusion information, ambient light information, diffuse reflection light information, and specular reflection light information.

[0138] As an example, suppose the ambient light information in the acquired lighting information is I. A ∈R H*W*3 The diffuse information is I. D ∈R H*W*3 The specular information is I. S ∈R H*W*3 The lighting information after adding the AO Map is corrected as follows:

[0139] I final =I A *AO+I D +I S ,

[0140] Among them, I final It is the corrected lighting information, that is, the RGB color of the current pixel.

[0141] The meanings of the letters R, H, and W are the same as above, and will not be repeated here.

[0142] V. Lighting Rendering.

[0143] like Figure 5 As shown in this embodiment, after correcting the acquired lighting information based on the AO Map (such as occlusion weighting), the aforementioned game screen image can be rendered with lighting.

[0144] 6. Follow-up processing.

[0145] like Figure 5 As shown in this embodiment, after rendering the aforementioned game screen image with lighting, other processing related to fragment coloring can be performed, such as color correction and image optimization. After fragment coloring is completed, the image is then displayed.

[0146] In summary, the embodiments disclosed herein can generate AO Maps in real time within the mobile GPU based on GBuffer. By using GBuffers such as Depth and Normal through a neural network to generate AO Maps, and applying them to global illumination, the rendering quality can be improved while ensuring real-time performance and reducing rendering power consumption.

[0147] To implement the above embodiments, this disclosure also proposes an image rendering apparatus.

[0148] Figure 6 This is a schematic diagram of the structure of an image rendering apparatus provided in an embodiment of the present disclosure.

[0149] like Figure 6 As shown, the image rendering device 900 may include: a processing module 610, a feature extraction module 620, a correction module 630, and a rendering module 640.

[0150] The processing module 610 is used to respond to user operations, acquire the target image to be rendered, and fuse the depth information and normal information of the target image to obtain fused information.

[0151] Feature extraction module 620 is used to extract features from the fused information to obtain ambient light occlusion information of the target image;

[0152] The correction module 630 is used to correct the illumination information of the target image based on the ambient light occlusion information to obtain the corrected illumination information.

[0153] The rendering module 640 is used to perform lighting rendering on the target image based on the corrected lighting information.

[0154] In some possible implementations, the feature extraction module 620 includes:

[0155] The downsampling unit is used to downsample the fused information to obtain the target fused information;

[0156] The processing unit is used to extract features from the target fusion information to obtain feature information, and to filter the target fusion information to obtain key information;

[0157] The first determining unit is used to determine ambient light occlusion information based on feature information and key information.

[0158] In some possible implementations, the processing unit is further configured to:

[0159] The target fusion information corresponding to the downsampling factor is input into the corresponding encoder in the neural network for feature extraction, and the feature information output by the corresponding encoder is obtained; wherein, at least two encoders in the neural network correspond to different downsampling factors.

[0160] In some possible implementations, the processing unit is further configured to:

[0161] The target fusion information, downsampled by a fixed factor, is input into the bottleneck layer of the neural network for information filtering to obtain key information.

[0162] In some possible implementations, the neural network further includes a multi-layer decoder, where the decoder corresponds to the encoder; the first determining unit is further configured to:

[0163] For the lowest-level target decoder, the key information and the feature information output by the encoder corresponding to the target decoder are input into the target decoder to obtain the candidate ambient light occlusion information output by the target decoder.

[0164] For any decoder other than the target decoder, the candidate ambient occlusion information output by the previous layer decoder and the feature information output by the encoder corresponding to the decoder are input into the decoder to obtain the candidate ambient occlusion information output by the decoder.

[0165] The candidate ambient occlusion information output by the top-level decoder is determined as ambient occlusion information.

[0166] In some possible implementations, the candidate ambient light occlusion information output by the target decoder is obtained by adjusting the ambient light occlusion information generated by the target decoder based on key information, and the target decoder generates ambient light occlusion information based on the feature information output by the encoder corresponding to the target decoder.

[0167] The candidate ambient occlusion information output by the decoder is obtained by adjusting the ambient occlusion information generated by the decoder based on the candidate ambient occlusion information output by the previous layer decoder. The decoder generates ambient occlusion information based on the feature information output by the encoder corresponding to the decoder.

[0168] In some possible implementations, the correction module 630 includes:

[0169] The acquisition unit, conventionally speaking, acquires the illumination information of the target image; wherein, the illumination information includes ambient light information, diffuse light information, and specular light information;

[0170] The second determining unit is used to determine the corrected illumination information based on ambient light occlusion information, ambient light information, diffuse reflection light information, and specular reflection light information.

[0171] In some possible implementations, the second determining unit is further configured to:

[0172] By fusing ambient light occlusion information with ambient light information, the target ambient light information is obtained;

[0173] The target ambient light information, diffuse light information, and specular light information are superimposed to obtain the corrected lighting information.

[0174] In some possible implementations, the second determining unit is further configured to:

[0175] The product of ambient light occlusion information and ambient light information is determined as the corrected ambient light information;

[0176] The sum of the corrected ambient light information, diffuse light information, and specular light information is determined as the corrected illumination information.

[0177] It should be noted that the foregoing explanation of the image rendering method embodiment also applies to the image rendering apparatus of this embodiment, and will not be repeated here.

[0178] The image rendering apparatus of this disclosure, in response to a user operation, acquires a target image to be rendered, and fuses the depth information and normal information of the target image to obtain fused information; it then extracts features from the fused information to obtain ambient occlusion information of the target image; based on the ambient occlusion information, it corrects the lighting information of the target image to obtain corrected lighting information; and finally, it renders the target image based on the corrected lighting information. Therefore, by using the depth information and normal information of the target image at the terminal to determine the ambient occlusion information of the target image and applying it to the lighting information of the target image, rendering quality can be improved while ensuring real-time performance and reducing rendering power consumption.

[0179] To implement the above embodiments, this disclosure also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the image rendering method as described in any of the foregoing embodiments.

[0180] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. For example, the electronic device 700 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0181] Reference Figure 7The electronic device 700 may include one or more of the following components: processing component 702, memory 704, power component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716.

[0182] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0183] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0184] Power component 706 provides power to various components of electronic device 700. Power component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0185] Multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0186] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0187] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0188] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0189] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0190] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0191] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0192] To implement the above embodiments, this disclosure also proposes a chip, wherein the chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to perform the image rendering method provided in any of the foregoing embodiments.

[0193] Figure 8 This is a schematic diagram of a chip structure provided in an embodiment of this disclosure. See also... Figure 8 The diagram shown is a schematic representation of the structure of chip 800, but is not limited to this.

[0194] Chip 800 includes processing circuitry 801, which is configured to execute any of the above image rendering methods.

[0195] In some embodiments, the chip 800 further includes one or more interface circuits 802. Optionally, the interface circuit 802 is connected to the memory 803, and the interface circuit 802 can be used to receive signals from the memory 803 or other devices, and the interface circuit 802 can be used to send signals to the memory 803 or other devices. For example, the interface circuit 802 can read instructions stored in the memory 803 and send the instructions to the processing circuit 801.

[0196] In some embodiments, the interface circuit 802 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 801 performs other steps.

[0197] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0198] In some embodiments, chip 800 further includes one or more memories 803 for storing instructions. Optionally, all or part of the memories 803 may be located outside of chip 800.

[0199] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image rendering method as described in any of the foregoing method embodiments.

[0200] To implement the above embodiments, this disclosure also proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the image rendering method as described in any of the foregoing method embodiments.

[0201] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0202] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0203] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0204] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and compact disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0205] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using at least one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0206] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0207] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0208] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image rendering method, characterized in that, include: In response to user operation, the target image to be rendered is acquired, and the depth information and normal information of the target image are fused to obtain fused information; Feature extraction is performed on the fused information to obtain the ambient light occlusion information of the target image; Based on the ambient light occlusion information, the illumination information of the target image is corrected to obtain the corrected illumination information; The target image is then rendered using the corrected lighting information.

2. The method according to claim 1, characterized in that, The step of extracting features from the fused information to obtain the ambient light occlusion information of the target image includes: The fused information is downsampled to obtain the target fused information; Feature extraction is performed on the target fusion information to obtain feature information, and information filtering is performed on the target fusion information to obtain key information; The ambient light occlusion information is determined based on the feature information and the key information.

3. The method according to claim 2, characterized in that, The step of extracting features from the target fusion information to obtain feature information includes: The target fusion information corresponding to the downsampling factor is input into the corresponding encoder in the neural network for feature extraction to obtain the feature information output by the corresponding encoder; wherein, at least two encoders in the neural network correspond to different downsampling factors.

4. The method according to claim 2, characterized in that, The process of filtering the target fused information to obtain key information includes: The target fusion information, downsampled by a fixed factor, is input into the bottleneck layer of the neural network for information filtering to obtain the key information.

5. The method according to claim 3, characterized in that, The neural network further includes a multi-layer decoder, which corresponds to the encoder; determining the ambient light occlusion information based on the feature information and the key information includes: For the lowest-level target decoder, the key information and the feature information output by the encoder corresponding to the target decoder are input into the target decoder to obtain the candidate ambient light occlusion information output by the target decoder. For any of the decoders other than the target decoder, the candidate ambient light occlusion information output by the previous layer decoder and the feature information output by the encoder corresponding to the decoder are input into the decoder to obtain the candidate ambient light occlusion information output by the decoder. The candidate ambient light occlusion information output by the top-level decoder is determined as the ambient light occlusion information.

6. The method according to claim 5, characterized in that, The candidate ambient light occlusion information output by the target decoder is obtained by adjusting the ambient light occlusion information generated by the target decoder based on the key information. The target decoder generates ambient light occlusion information based on the feature information output by the encoder corresponding to the target decoder. The candidate ambient light occlusion information output by the decoder is obtained by adjusting the ambient light occlusion information generated by the decoder based on the candidate ambient light occlusion information output by the previous layer decoder. The decoder generates ambient light occlusion information based on the feature information output by the encoder corresponding to the decoder.

7. The method according to claim 1, characterized in that, The step of correcting the illumination information of the target image based on the ambient light occlusion information to obtain the corrected illumination information includes: Acquire the illumination information of the target image; wherein, the illumination information includes ambient light information, diffuse light information, and specular light information; The corrected illumination information is determined based on the ambient light occlusion information, the ambient light information, the diffuse reflection light information, and the specular reflection light information.

8. The method according to claim 7, characterized in that, The step of determining the corrected illumination information based on the ambient light occlusion information, the ambient light information, the diffuse reflection light information, and the specular reflection light information includes: The ambient light occlusion information is fused with the ambient light information to obtain the target ambient light information; The target ambient light information, the diffuse reflection light information, and the specular reflection light information are superimposed to obtain the corrected illumination information.

9. The method according to claim 7, characterized in that, The step of determining the corrected illumination information based on the ambient light occlusion information, the ambient light information, the diffuse reflection light information, and the specular reflection light information includes: The product of the ambient light occlusion information and the ambient light information is determined as the corrected ambient light information; The sum of the corrected ambient light information, the diffuse light information, and the specular light information is determined as the corrected illumination information.

10. An image rendering apparatus, characterized in that, The device includes: The processing module is used to respond to user operations, acquire the target image to be rendered, and fuse the depth information and normal information of the target image to obtain fused information; The feature extraction module is used to extract features from the fused information to obtain the ambient light occlusion information of the target image; The correction module is used to correct the illumination information of the target image based on the ambient light occlusion information to obtain the corrected illumination information. The rendering module is used to perform lighting rendering on the target image based on the corrected lighting information.

11. The apparatus according to claim 10, characterized in that, The feature extraction module includes: A downsampling unit is used to downsample the fused information to obtain target fused information; The processing unit is used to extract features from the target fusion information to obtain feature information, and to filter information from the target fusion information to obtain key information; The first determining unit is used to determine the ambient light occlusion information based on the feature information and the key information.

12. The apparatus according to claim 11, characterized in that, The processing unit is further configured to: The target fusion information corresponding to the downsampling factor is input into the corresponding encoder in the neural network for feature extraction to obtain the feature information output by the corresponding encoder; wherein, at least two encoders in the neural network correspond to different downsampling factors.

13. The apparatus according to claim 11, characterized in that, The processing unit is further configured to: The target fusion information, downsampled by a fixed factor, is input into the bottleneck layer of the neural network for information filtering to obtain the key information.

14. The apparatus according to claim 12, characterized in that, The neural network further includes a multi-layer decoder, which corresponds to the encoder; the first determining unit is further configured to: For the lowest-level target decoder, the key information and the feature information output by the encoder corresponding to the target decoder are input into the target decoder to obtain the candidate ambient light occlusion information output by the target decoder. For any of the decoders other than the target decoder, the candidate ambient light occlusion information output by the previous layer decoder and the feature information output by the encoder corresponding to the decoder are input into the decoder to obtain the candidate ambient light occlusion information output by the decoder. The candidate ambient light occlusion information output by the top-level decoder is determined as the ambient light occlusion information.

15. The apparatus according to claim 10, characterized in that, The correction module includes: An acquisition unit is used to acquire the illumination information of the target image; wherein the illumination information includes ambient light information, diffuse reflection light information, and specular reflection light information; The second determining unit is used to determine the corrected illumination information based on the ambient light occlusion information, the ambient light information, the diffuse reflection light information, and the specular reflection light information.

16. The apparatus according to claim 15, characterized in that, The second determining unit is further configured to: The product of the ambient light occlusion information and the ambient light information is determined as the corrected ambient light information; The sum of the corrected ambient light information, the diffuse light information, and the specular light information is determined as the corrected illumination information.

17. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 9.

18. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method as described in any one of claims 1 to 9.

19. A chip, characterized in that, The chip includes an interface circuit and a processing circuit that are coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is used to implement the steps of the method as described in any one of claims 1 to 9.

20. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 9.

Images