Soft shadow image rendering method and device, equipment, storage medium and program product

Abstract

The invention provides a soft shadow image rendering method and device, electronic equipment, a computer readable storage medium and a computer program product, and the method comprises the steps: obtaining a to-be-rendered image which comprises a shadow region; generating a shadow mask corresponding to the to-be-rendered image based on the shadow area in the to-be-rendered image; obtaining point cloud data corresponding to the to-be-rendered image, and generating a soft shadow mask corresponding to the to-be-rendered image based on the point cloud data and the shadow mask; and performing image rendering on the to-be-rendered image based on the soft shadow mask to obtain a soft shadow image corresponding to the to-be-rendered image. Therefore, the rendering efficiency and the rendering effect of the soft shadow image can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for rendering soft shadow images. Background Technology

[0002] When rendering soft shadow images, most related technologies reconstruct the shadow image using geometric samples and the shadow mask after generating the corresponding shadow mask. However, this method requires massive computation, such as using sufficient samples to design 3D and color, making it difficult to achieve real-time rendering of shadow reconstruction. It not only has extremely high performance overhead, but also has low efficiency when rendering soft shadow images. At the same time, the method of reconstructing the shadow image using only geometric samples and the shadow mask will also result in a rather harsh shadow effect in the generated soft shadow image, reducing the rendering effect when rendering soft shadow images. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for rendering soft shadow images, which can improve the efficiency and rendering effect when rendering soft shadow images.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for rendering soft shadow images, the method comprising:

[0006] Obtain an image to be rendered, which includes a shadow area;

[0007] Based on the shadow region in the image to be rendered, generate a shadow mask corresponding to the image to be rendered;

[0008] Obtain the point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask;

[0009] The image to be rendered is rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

[0010] This application provides a rendering apparatus for soft shadow images, the apparatus comprising:

[0011] The first acquisition module is used to acquire an image to be rendered, wherein the image to be rendered includes a shadow area;

[0012] The generation module is used to generate a shadow mask corresponding to the image to be rendered based on the shadow region in the image to be rendered;

[0013] The second acquisition module is used to acquire point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask.

[0014] The rendering module is used to render the image to be rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

[0015] In the above scheme, the generation module is further configured to obtain the brightness value of each pixel in the image to be rendered; for each pixel in the image to be rendered, the following processing is performed: dilating the pixel to obtain the local maximum brightness value of the pixel, and eroding the pixel to obtain the local minimum brightness value of the pixel; determining the mask value of the pixel based on the local maximum brightness value and the local minimum brightness value; and obtaining the shadow mask corresponding to the image to be rendered based on the mask value of each pixel.

[0016] In the above scheme, the device further includes a third acquisition module, which is used to acquire the target neighborhood range of the pixel, the target neighborhood range including at least one adjacent pixel adjacent to the pixel; the generation module is further used to select the maximum brightness value from the brightness values ​​of each of the adjacent pixels and use the maximum brightness value as the local maximum brightness value of the pixel; and to select the minimum brightness value from the brightness values ​​of each of the adjacent pixels and use the minimum brightness value as the local minimum brightness value of the pixel.

[0017] In the above scheme, the generation module is further configured to obtain the ratio of the minimum brightness value and the local maximum brightness value of the pixel; when the ratio is less than the target ratio, a first constant is determined as the mask value of the pixel; when the ratio is greater than or equal to the target ratio, a second constant is determined as the mask value of the pixel; wherein, the first constant and the second constant are different.

[0018] In the above scheme, the second acquisition module is further used to acquire the depth image of the image to be rendered and the coordinates of each pixel in the image to be rendered; based on the depth image, the coordinates of each pixel in the image to be rendered are transformed to obtain the three-dimensional coordinates of each pixel; based on the three-dimensional coordinates of each pixel in the image to be rendered, the point cloud data corresponding to the image to be rendered is obtained.

[0019] In the above scheme, the device further includes an elimination module, which is used to perform shadow elimination on the shadow area in the image to be rendered based on the shadow mask to obtain a shadowless image; the rendering module is further used to fuse the soft shadow mask with the shadowless image to obtain a fused image; and to render the fused image to obtain a soft shadow image corresponding to the image to be rendered.

[0020] In the above scheme, the elimination module is further configured to determine the shadow region in the image to be rendered based on the shadow mask; and to perform at least one denoising process on the image to be rendered based on the shadow region to obtain the shadow-free image.

[0021] In the above scheme, the generation module is further configured to predict the shadow level of each pixel in the image to be rendered based on the point cloud data and the shadow mask, and obtain the shadow level value of each pixel; and obtain the soft shadow mask corresponding to the image to be rendered based on the shadow level value of each pixel.

[0022] In the above scheme, the device further includes a first parsing module, which is used to parse the image to be rendered to obtain the illumination direction of the virtual light source corresponding to the image to be rendered; the generation module is further used to generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data, the shadow mask, and the illumination direction of the virtual light source.

[0023] In the above scheme, the device further includes a second parsing module, which is used to parse the image to be rendered to obtain the direct light component and the scattered light component corresponding to the image to be rendered; the generation module is further used to generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data, the shadow mask, the direct light component and the scattered light component.

[0024] In the above scheme, the generation module is further configured to determine the potential shadow region in the image to be rendered based on the direct light component and the scattered light component; predict the shadow degree of each pixel in the image to be rendered based on the point cloud data, the shadow mask, and the potential shadow region in the image to be rendered, and obtain the shadow degree value of each pixel; and obtain the soft shadow mask corresponding to the image to be rendered based on the shadow degree value of each pixel.

[0025] In the above scheme, the generation module is further configured to perform the following processing for each pixel in the image to be rendered: based on the direct light component and the scattered light component, obtain a first brightness value of the pixel in the direct light dimension corresponding to the direct light component and a second brightness value in the scattered light dimension corresponding to the scattered light component; based on the first brightness value and the second brightness value, select a target pixel from the plurality of pixels included in the image to be rendered; wherein, the first brightness value of the target pixel is less than a first brightness value threshold, and the second brightness value of the target pixel is greater than a second brightness value threshold, and the second brightness value threshold is greater than the first brightness value threshold; determine the area where the target pixel is located in the image to be rendered as the potential shadow area in the image to be rendered.

[0026] This application provides an electronic device, including:

[0027] Memory, used to store executable instructions;

[0028] The processor, when executing executable instructions stored in the memory, implements the soft shadow image rendering method provided in the embodiments of this application.

[0029] This application provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor will execute the soft shadow image rendering method provided in this application.

[0030] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the soft shadow image rendering method provided in this application.

[0031] The embodiments of this application have the following beneficial effects:

[0032] First, a shadow mask corresponding to the image to be rendered, including the shadow region, is generated. Then, based on the point cloud data of the image to be rendered and the shadow mask, a soft shadow mask corresponding to the image to be rendered is generated. Finally, the image to be rendered is rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered. Thus, obtaining the soft shadow image by rendering the image using the generated soft shadow mask not only reduces performance overhead but also improves the efficiency of rendering soft shadow images. Furthermore, since shadow generation is largely related to the three-dimensionality of objects, this application also utilizes the point cloud data of the image to be rendered during the soft shadow image rendering process, that is, taking into account the three-dimensionality of objects in the image to be rendered in a three-dimensional environment, thereby improving the rendering effect of the soft shadow image. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the architecture of the soft shadow image rendering system 100 provided in the embodiments of this application;

[0034] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0035] Figure 3 This is a flowchart illustrating the soft shadow image rendering method provided in the embodiments of this application;

[0036] Figure 4 This is a schematic diagram of the process for generating the shadow mask corresponding to the image to be rendered, provided in an embodiment of this application.

[0037] Figure 5 This is a schematic diagram of the target neighborhood range of a pixel provided in an embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the image to be rendered and the corresponding depth image provided in the embodiments of this application;

[0039] Figure 7 This is a schematic diagram of the model structure of the depth image generation model provided in the embodiments of this application;

[0040] Figure 8 This is a flowchart illustrating the soft shadow image rendering method provided in the embodiments of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Client, also known as user terminal, refers to the program that provides local services to users in contrast to the server. Except for some applications that can only run locally, it is generally installed on ordinary client machines and needs to cooperate with the server to run. That is, there needs to be a corresponding server and service program in the network to provide the corresponding services. Thus, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application.

[0047] 2) Render Target (RT): The video memory buffer, or memory area, used to render pixels. Multiple such memory areas can exist at the same time, meaning multiple render targets can exist simultaneously.

[0048] 3) Depth image: Depth refers to the Z coordinate of a pixel in the camera coordinate system of the view. It is a unit of space. A depth map can be obtained for any image. A depth map is a two-dimensional image that stores the depth values ​​of all pixels in a single view. It is a unit of space, such as millimeters. The significance of a depth map is to express the three-dimensional results of image matching in an orderly manner with less storage space.

[0049] 4) Virtual light source is a "light source" set up in computer animation software or virtual engine. The role of virtual light source in representing the viewpoint during animation production is equivalent to that of physical light source in the traditional sense. The objects illuminated by virtual light source and physical light source are completely different, but their functions are extremely similar. Physical light source illuminates real people or actual built scenes, while virtual light source illuminates models built in 3D software, which can realize infinite possibilities.

[0050] 5) A virtual scene is a virtual scene displayed (or provided) by an application when it runs on a terminal. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move within this virtual scene.

[0051] 6) Virtual light refers to the light emitted by a virtual light source used to illuminate a virtual scene. Virtual light includes direct light and indirect light. Direct light is emitted by a virtual light source and reflected by a virtual lighting point to a virtual camera. Indirect light is emitted by a virtual light source, reflected at least once to a virtual lighting point, and finally reflected by the virtual lighting point to a virtual camera.

[0052] 7) A virtual camera is a "camera" set up in computer animation software or a virtual engine. In animation, the virtual camera's role in representing the viewpoint is equivalent to a traditional camera. While the subjects of a virtual camera and a physical camera are completely different, their functions are extremely similar. A physical camera captures real people or actual, constructed scenes, while a virtual camera captures models built in 3D software, allowing for limitless possibilities. Virtual cameras are presented as icons in the virtual engine and also have parameters such as lens, focal length, focus, aperture, and depth of field. They can perform camera actions such as "push, pull, pan, tilt, track, flick, rise, fall, and combined movements," achieving shooting effects that are difficult or impossible for physical cameras, such as passing through walls, keyholes, or objects. The parameters that need to be adjusted for a physical camera are distributed on the camera body and require manual operation. The camera parameters of a virtual camera are integrated into buttons or numerical input fields on the panel. The operator only needs to input parameters or drag the mouse. Sometimes a few keyframes can determine the motion path of the virtual camera. In actual shooting, physical cameras often need stabilizers or motion control systems, and even then, the image shake still exists.

[0053] 8) Soft shadows refer to the soft, gradual shadows created at the edges of objects due to factors such as the size of the light source, the scattering of light, and the obstruction of objects. Unlike hard shadows, soft shadows have softer, smoother edges and are closer to the shadow effects under realistic lighting conditions.

[0054] 9) Hard shadows refer to the distinct and clearly defined shadow effects produced when there is no obstruction between an object and the light source under illumination. Hard shadows are characterized by sharp, clear edges in the shadow area, without any gradation or blurring, appearing as the edge of an object completely blocking the light source.

[0055] 10) Soft shadow masks, typically grayscale or transparency images, are used to describe the shadow effects produced when a light source shines on an object's surface. Soft shadow masks can enhance the realism and lifelikeness of images, making lighting effects appear more natural and softer. A soft shadow mask usually contains transparency or grayscale information about the shadows on the object's surface, reflecting the position and intensity of the light source and the relative positions of objects. By applying a soft shadow mask, it's possible to achieve a gradual fading or lightening effect on the object's surface, rather than simply creating black and white shadows.

[0056] 11) Potential shadow areas refer to areas that may be covered by shadows under illumination. In other words, based on the position of the light source and the direction of light propagation, these areas may be obstructed by other objects or surfaces, resulting in a shadow effect. When light propagates from a light source to an object's surface, if the light is blocked by other objects, preventing it from directly reaching a certain area, then that area will likely have a potential shadow. Considering potential shadow areas can help simulate real-world lighting effects, making object rendering more realistic and lifelike. By analyzing and processing potential shadow areas, the propagation path of light can be accurately simulated in rendering, and appropriate shadow effects can be determined based on the object's geometry and the position of the light source. This helps enhance the realism and three-dimensionality of the rendered image, making the lighting effects more natural and lifelike.

[0057] See Figure 1 , Figure 1This is a schematic diagram of the architecture of the soft shadow image rendering system 100 provided in this application embodiment. To realize the soft shadow image rendering scene (for example, this application embodiment can be applied in a game scene, that is, the soft shadow image rendering scene can be that during the user's game, the server obtains the game scene image in the game, the game scene image includes shadow areas; based on the shadow areas in the game scene image, a shadow mask corresponding to the game scene image is generated; point cloud data corresponding to the game scene image is obtained, and based on the point cloud data and the shadow mask, a soft shadow mask corresponding to the game scene image is generated; based on the soft shadow mask, the game scene image is rendered to obtain the soft shadow image corresponding to the game scene image), the terminal (terminal 400 is shown as an example) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of both. The terminal 400 is used for the user to use the client 401 and display on the display interface (display interface 401-1 is shown as an example). The terminal 400 and the server 200 are connected to each other through a wired or wireless network.

[0058] The server 200 is configured to: acquire an image to be rendered, which includes a shadow region; generate a shadow mask corresponding to the image to be rendered based on the shadow region; acquire point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask; render the image to be rendered based on the soft shadow mask to obtain a soft shadow image corresponding to the image to be rendered; and send the soft shadow image corresponding to the image to be rendered to the terminal 400.

[0059] Terminal 400 is also used to receive the soft shadow image corresponding to the image to be rendered, and to display the soft shadow image based on the display interface.

[0060] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, set-top box, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices, smart speakers, and smartwatches), but is not limited to these. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0061] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. In practical applications, the electronic device can be... Figure 1 The server 200 or terminal 400 shown are described in the following document. Figure 2 , Figure 2 The illustrated electronic device includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0062] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0063] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0064] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0065] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0066] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0067] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0068] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0069] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0070] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0071] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A rendering apparatus 455 storing a soft shadow image in memory 450 is shown. This apparatus can be software in the form of programs and plugins, and includes the following software modules: a first acquisition module 4551, a generation module 4552, a second acquisition module 4553, and a rendering module 4554. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0072] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the soft shadow image rendering apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the soft shadow image rendering method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0073] In some embodiments, the terminal or server can implement the soft shadow image rendering method provided in this application embodiment by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP or a web browser APP; it can also be a mini-program, that is, a program that only needs to be downloaded into the browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0074] Based on the above description of the soft shadow image rendering system and electronic device provided in the embodiments of this application, the soft shadow image rendering method provided in the embodiments of this application is described below. In actual implementation, the soft shadow image rendering method provided in the embodiments of this application can be implemented by the terminal or the server alone, or by the terminal and the server working together, so that... Figure 1 The following description uses the example of server 200 executing the soft shadow image rendering method provided in this application embodiment. See also... Figure 3 , Figure 3 This is a schematic flowchart of the soft shadow image rendering method provided in the embodiments of this application. Below, it will be combined with... Figure 3 The steps shown are explained.

[0075] Step 101: The server obtains the image to be rendered, which includes shadow areas.

[0076] In practice, the image to be rendered can be pre-stored locally, obtained from external sources (such as the Internet), or captured in real time, for example, by a camera. This application does not limit the scope of the embodiments.

[0077] It should be noted that the image to be rendered can be an image corresponding to a virtual scene during a game, or an image corresponding to an animation scene or anime scene during video playback. This application does not limit this aspect.

[0078] Step 102: Generate a shadow mask for the image to be rendered based on the shadow regions in the image to be rendered.

[0079] It should be noted that after obtaining the image to be rendered, the shadow areas in the image to be rendered are detected, and a shadow mask corresponding to the image to be rendered is generated based on the detected shadow areas. The shadow mask is a binary image used to distinguish between shadow areas and non-shadow areas in the image to be rendered; for example, the pixel value of the shadow area in the image to be rendered is identified as 1, and the pixel value of the non-shadow area is identified as 0, or the pixel value of the shadow area is identified as 1, and the pixel value of the non-shadow area is identified as 0. This embodiment of the application does not limit this.

[0080] In actual implementation, see Figure 4 , Figure 4 This is a schematic diagram of the process for generating the shadow mask corresponding to the image to be rendered, provided in an embodiment of this application. Figure 4 The process of generating the shadow mask corresponding to the image to be rendered based on the shadow area in the image to be rendered, i.e., step 102, can be achieved through the following steps.

[0081] Step 1021: Obtain the brightness value of each pixel in the image to be rendered.

[0082] In practice, the image to be rendered is converted into RGB format image data. Specifically, the color value of each pixel in the image to be rendered is obtained in the three color channels R, G, and B, i.e., red, green, and blue channels. Then, based on the RGB format image data, the brightness value of each pixel in the image to be rendered is determined. Specifically, for each pixel, the following processing is performed: the color values ​​of the pixel in the three color channels are summed to obtain the summation result, and then the ratio of the summation result to a third constant is obtained. This ratio is used as the brightness value of the pixel.

[0083] V = (R + G + B) / 3 ... Formula (1);

[0084] Where V indicates the brightness value of the pixel, R indicates the color value of the pixel in the red channel, G indicates the color value of the pixel in the green channel, B indicates the color value of the pixel in the blue channel, and 3 is a preset third constant.

[0085] Step 1022: For each pixel in the image to be rendered, perform the following processing: dilate the pixel to obtain the local maximum brightness value of the pixel, and erode the pixel to obtain the local minimum brightness value of the pixel; determine the mask value of the pixel based on the local maximum brightness value and the local minimum brightness value.

[0086] In actual implementation, before performing dilation processing on the pixel to obtain the local maximum brightness value of the pixel, the target neighborhood range of the pixel is obtained. The target neighborhood range includes at least one adjacent pixel that is adjacent to the pixel.

[0087] It should be noted that the target neighborhood range here is preset. For example, it can be the area formed by 8 neighboring pixels around a pixel, or the area formed by 15 neighboring pixels around a pixel, etc. This application embodiment does not limit this.

[0088] For example, see Figure 5 , Figure 5 This is a schematic diagram of the target neighborhood range of a pixel provided in an embodiment of this application, based on Figure 5 For the 9 pixels indicated by dashed box 501, when the target neighborhood range indicates the area formed by the 8 adjacent pixels around the pixel, the 8 white pixels around the black pixel indicated by 5011 are the adjacent pixels of the black pixel, that is, the area formed by the white pixels is the target neighborhood range of the black pixel; and for the 16 pixels indicated by dashed box 502, when the target neighborhood range indicates the area formed by the 15 adjacent pixels around the pixel, the 15 white pixels around the black pixel indicated by 5021 are the adjacent pixels of the black pixel, that is, the area formed by the white pixels is the target neighborhood range of the black pixel.

[0089] Then, after obtaining the target neighborhood range of a pixel, the process of dilating the pixel to obtain its local maximum brightness value can be achieved by selecting the maximum brightness value from the brightness values ​​of all neighboring pixels and using this maximum brightness value as the local maximum brightness value of the pixel.

[0090] M(i,j)=max V(i+m,j+n)……Formula (2);

[0091]

[0092]

[0093] Where V is the brightness value of a pixel, i and j are the row and column coordinates of a pixel, K is the size of the kernel, which indicates the number of adjacent pixels, and m and n are the offset of the kernel, which is the sliding range of the kernel on the image.

[0094] It's important to note that dilation is a mathematical morphological dilation transformation. Through dilation, the brightness value of each pixel is replaced with the maximum brightness value in its surrounding neighborhood, making the brightness values ​​in the image brighter or more prominent. Specifically, dilation typically involves a structuring element (also called a kernel or template) that defines the shape and size of the dilation operation. This structuring element slides across the image (usually from left to right and top to bottom), and at each location, it determines the neighborhood range. Then, it finds the maximum brightness value in this neighborhood and assigns that value to the pixel at the center of the structuring element. Thus, after dilation, the value of each pixel in the brightness channel of the original image is the maximum brightness value in its neighborhood. This can be used to emphasize brightness variations in an image or highlight details in local areas.

[0095] The process of eroding pixels to obtain the local minimum brightness value of a pixel can be achieved by selecting the minimum brightness value from the brightness values ​​of all adjacent pixels and using this minimum brightness value as the local minimum brightness value of the pixel.

[0096] m(i,j)=min V(i+m,j+n)...Formula (5);

[0097]

[0098]

[0099] Where V is the brightness value of a pixel, i and j are the row and column coordinates of a pixel, K is the size of the kernel, which indicates the number of adjacent pixels, and m and n are the offset of the kernel, which is the sliding range of the kernel on the image.

[0100] It's important to note that erosion, also known as mathematical morphological erosion, is used to extract localized areas of minimum brightness in an image. Through erosion, the brightness value of each pixel is replaced by the minimum brightness value within its surrounding neighborhood. Specifically, erosion typically involves a structuring element, as described earlier, which defines the shape and size of the erosion operation. This structuring element slides across the image (usually from left to right and top to bottom), and at each location, it determines the neighborhood range. It then finds the minimum brightness value within this neighborhood and assigns it to the pixel at the center of the structuring element. Thus, after erosion, the value of each pixel in the original image's brightness channel is the minimum brightness value within its neighborhood. This can also be used to emphasize brightness variations in an image or highlight detailed features in certain local areas.

[0101] In actual implementation, after determining the local maximum brightness value and the local minimum brightness value, the process of determining the mask value of the pixel based on the local maximum brightness value and the local minimum brightness value can be as follows: obtain the ratio of the minimum brightness value of the pixel to the local maximum brightness value; when the ratio is less than the target ratio, determine the first constant as the mask value of the pixel; when the ratio is greater than or equal to the target ratio, determine the second constant as the mask value of the pixel; wherein, the first constant and the second constant are different.

[0102] It should be noted that the target ratio can be preset, such as 0.5, and the first constant and the second constant can also be preset, for example, the first constant can be 0 and the second constant can be 1; however, the embodiments of this application do not limit this.

[0103] For example, the process of determining the mask value of a pixel based on the local maximum and local minimum brightness values, i.e.

[0104]

[0105] Where m(i,j) is the local minimum brightness value of the pixel, M(i,j) is the local maximum brightness value of the pixel, T is the pre-set target ratio, and i and j are the row coordinates and column coordinates of the pixel.

[0106] Thus, determining the shadow mask based on the local minimum and maximum brightness values ​​of pixels helps to distinguish the brightness variation areas in the image to be rendered, thereby making it easier to differentiate between shadow and non-shadow areas in the image to be rendered, and thus improving the rendering effect of soft shadow images in subsequent processes.

[0107] Step 1023: Based on the mask values ​​of each pixel, obtain the shadow mask corresponding to the image to be rendered.

[0108] In actual implementation, as mentioned above, the shadow mask is a binary image. Therefore, after determining the mask value of each pixel, that is, 0 or 1, the binary image corresponding to the image to be rendered, that is, the shadow mask, can be obtained.

[0109] Step 103: Obtain the point cloud data corresponding to the image to be rendered, and generate the soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask.

[0110] In practice, the process of obtaining the point cloud data corresponding to the image to be rendered can be as follows: obtain the depth image of the image to be rendered and the coordinates of each pixel in the image to be rendered; based on the depth image, transform the coordinates of each pixel in the image to be rendered to obtain the three-dimensional coordinates of each pixel; based on the three-dimensional coordinates of each pixel in the image to be rendered, obtain the point cloud data corresponding to the image to be rendered.

[0111] In some embodiments, the process of obtaining the depth image corresponding to the image to be rendered specifically involves: constructing a spatial coordinate system corresponding to the image to be rendered; determining the height value, i.e., the depth value, of each pixel in the image to be rendered in the spatial coordinate system based on the image to be rendered; and generating the depth image corresponding to the image to be rendered based on the height values. For example, see [link to example]. Figure 6 , Figure 6 This is a schematic diagram of the image to be rendered and the corresponding depth image provided in the embodiments of this application, based on... Figure 6 601 shows the image to be rendered, and 602 shows the depth image corresponding to the image to be rendered. First, the image to be rendered as shown in 601 is obtained, and a spatial coordinate system corresponding to the image to be rendered is constructed. Based on the image to be rendered, the height value of each pixel in the image to be rendered in the spatial coordinate system is determined. Among them, the height values ​​of each pixel in the gray part of the image content indicated by the dashed box 6021 are the same, and are different from the height values ​​of each pixel in the black part indicated by 602. Thus, based on the height values ​​of each pixel, the depth image corresponding to the image to be rendered as shown in 602 is generated.

[0112] In other embodiments, the process of obtaining the depth image corresponding to the image to be rendered can also be implemented by a model. Specifically, a pre-trained depth image generation model is obtained, the image to be rendered is input into the depth image generation model, the depth value of each pixel in the image to be rendered is obtained, and the depth image of the image to be rendered is obtained based on the depth value of each pixel.

[0113] It should be noted that, see Figure 7 , Figure 7 This is a schematic diagram of the model structure of the depth image generation model provided in the embodiments of this application, based on Figure 7Depth image generation models can use different architectures and loss functions. The size of the depth image is the same as the size of the image to be rendered, i.e., H×W, where H is the height of the image and W is the width of the image. Each pixel value D(i,j) in the depth image represents the distance from the pixel in the i-th row and j-th column of the image to the virtual camera corresponding to the image to be rendered. Specifically, the formula for the depth image generation model can be expressed as follows:

[0114] D=f(I, θ)...Formula (9);

[0115] Where I is the input image to be rendered, D is the output depth image, f is the function corresponding to the depth image generation model, and θ is the model parameter of the depth image generation model.

[0116] In practical implementation, for depth images, the process of transforming the coordinates of each pixel in the image to be rendered to obtain the 3D coordinates of each pixel involves several steps. First, the intrinsic parameter matrix of the virtual camera corresponding to the image to be rendered is obtained. This intrinsic parameter matrix is ​​typically a 3×3 matrix containing the virtual camera's focal length, principal point, and distortion parameters, which can usually be obtained through the virtual camera calibration process. Then, combining the intrinsic parameter matrix and the depth value of each pixel in the depth image, the coordinates of each pixel in the image to be rendered are transformed to obtain the 3D coordinates of each pixel. This transformation can be achieved by multiplying the coordinates of each pixel in the image to be rendered by its corresponding depth value to obtain the 3D coordinates of each pixel in the coordinate system corresponding to the virtual camera. Finally, the inverse of the intrinsic parameter matrix of the virtual camera is multiplied by the coordinates of each pixel to obtain the 3D coordinates of each pixel in the world coordinate system, i.e., the 3D coordinates of each pixel. The specific formula is as follows:

[0117]

[0118] Among them, K -1 It is the inverse of the camera intrinsic parameter matrix, where D(i,j) is the depth value of the pixel, and i and j are the row and column coordinates of the pixel, respectively. These are the coordinates of each pixel in the image to be rendered, i.e., two-dimensional coordinates.

[0119] In some embodiments, after obtaining the image to be rendered, the coordinates of the pixels in the image to be rendered can be normalized. Specifically, the coordinates of each pixel are normalized to obtain the normalized coordinates of each pixel in the image to be rendered; based on the depth image and the normalized coordinates, the coordinates of each pixel in the image to be rendered are transformed to obtain the three-dimensional coordinates of each pixel.

[0120] It should be noted that the process of normalizing the coordinates of each pixel to obtain the normalized coordinates of each pixel in the image to be rendered is, in other words,

[0121]

[0122]

[0123] Where i is the x-coordinate of a pixel, j is the y-coordinate of a pixel, H is the height of the image, and W is the width of the image. Thus, the pixel coordinates (i, j) in the image to be rendered are converted into normalized homogeneous coordinates (x, y, 1), where x and y are both in the range [-1, 1]. This eliminates the influence of image size and resolution on the coordinate transformation, making the coordinate transformation depend only on the camera's intrinsic parameter matrix.

[0124] In practice, the process of obtaining the point cloud data corresponding to the image to be rendered, based on the 3D coordinates of each pixel in the image, specifically involves obtaining a target matrix based on the 3D coordinates. This target matrix is ​​then used as the point cloud data corresponding to the image to be rendered. For example, the 3D coordinates can be stored in an N×M matrix, where N = H×W is the total number of pixels in the image to be rendered, and M is the dimension of the 3D coordinates. In this way, the point cloud data of the 3D scene corresponding to the image to be rendered is obtained, which can be used for subsequent operations such as shadow generation and rendering.

[0125] In practice, after acquiring point cloud data and shadow masks, a soft shadow mask can be generated based on the point cloud data and shadow masks. Specifically, the process of generating a soft shadow mask corresponding to the image to be rendered based on point cloud data and shadow masks can be as follows: based on point cloud data and shadow masks, the shadow intensity of each pixel in the image to be rendered is predicted to obtain the shadow intensity value of each pixel; based on the shadow intensity value of each pixel, the soft shadow mask corresponding to the image to be rendered is obtained.

[0126] It should be noted that the shadow level value of each pixel can indicate the shadow intensity of the pixel, that is, the transparency of the pixel. As mentioned above, a soft shadow mask is usually a grayscale image or a transparency image. Therefore, after determining the shadow level value of each pixel, that is, the transparency, we can obtain the transparency image corresponding to the image to be rendered, that is, the soft shadow mask.

[0127] It should be noted that the process of predicting the shadow intensity of each pixel in the image to be rendered, based on point cloud data and shadow masks, can be implemented by a shadow intensity prediction model. Specifically, the shadow intensity prediction model includes a point cloud encoder and a shadow predictor. First, a point cloud encoder is used to extract features from the point cloud data to obtain point cloud features, i.e.

[0128] F p =p(P, θ) p )...Formula (13);

[0129] Where P is the input point cloud data, p is the point cloud encoder, and θ p These are the parameters of the point cloud encoder.

[0130] Then, using a shadow predictor, based on point cloud features and a shadow mask, the shadow intensity value of each pixel is predicted, i.e.

[0131]

[0132] Where q is the shadow predictor, θ q These are the parameters of the shadow predictor, where S is the shadow mask and F is the shadow predictor. p It refers to the point cloud features obtained by extracting features from point cloud data.

[0133] In practice, in addition to generating soft shadow masks directly based on point cloud data and shadow masks, soft shadow masks can also be generated based on other parameters. Next, we will explain the process of generating soft shadow masks using specific other parameters as examples.

[0134] In some embodiments, the soft shadow mask can also be generated by combining the illumination direction of the virtual light emitted by the virtual light source. Specifically, after acquiring the image to be rendered, the image to be rendered can be parsed to obtain the illumination direction of the virtual light source corresponding to the image to be rendered. Thus, the process of generating the soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask can be as follows: the soft shadow mask corresponding to the image to be rendered is generated based on the point cloud data, the shadow mask, and the illumination direction of the virtual light source.

[0135] It should be noted that shadow formation is related to light. Based on this, the image to be rendered is analyzed to obtain the virtual light source corresponding to the image. Then, based on this virtual light source, the image to be rendered is analyzed to obtain the illumination direction of the virtual light source corresponding to the image. In this way, the soft shadow mask is generated by combining the illumination direction of the virtual light emitted by the virtual light source corresponding to the image. This enriches the data used in generating the soft shadow mask, which not only improves the accuracy of the generated soft shadow mask, but also improves the rendering effect of the subsequent soft shadow image. That is, it can generate realistic soft shadow effects, thereby enhancing the lighting and atmosphere of the soft shadow image.

[0136] It should be noted that the process of generating the soft shadow mask corresponding to the image to be rendered based on point cloud data, shadow mask, and the illumination direction of the virtual light source can also be achieved through a shadow intensity prediction model. Specifically, as shown in formula (13), firstly, a point cloud encoder is used to extract features from the point cloud data to obtain point cloud features. Then, a shadow predictor is used to predict the shadow intensity value of each pixel based on the point cloud features, shadow mask, and illumination direction of the virtual light source.

[0137]

[0138] Where q is the shadow predictor, θ q These are the parameters of the shadow predictor, where S is the shadow mask and F is the shadow predictor. p It is the point cloud feature obtained by feature extraction from point cloud data, where L is the illumination direction of the virtual light source.

[0139] It should be noted that after obtaining the predicted shadow level value of each pixel, the soft shadow mask corresponding to the image to be rendered can be obtained based on the shadow level value of each pixel. At the same time, the illumination direction of the virtual light source in the above process is obtained by analysis. In addition, the illumination direction of the virtual light source can also be preset. This application does not limit this.

[0140] In some embodiments, a soft shadow mask can be generated by combining the direct light component and the scattered light component corresponding to the image to be rendered. Specifically, after obtaining the image to be rendered, the image to be rendered can be parsed to obtain the direct light component and the scattered light component corresponding to the image to be rendered. Thus, the process of generating a soft shadow mask corresponding to the image to be rendered based on point cloud data and shadow mask can be as follows: generate a soft shadow mask corresponding to the image to be rendered based on point cloud data, shadow mask, direct light component and scattered light component.

[0141] It should be noted that, firstly, the image to be rendered is analyzed to obtain the direct light component and the scattered light component corresponding to the image to be rendered. That is, the direct light component and the scattered light component are separated from the image to be rendered. Here, different image processing techniques are used to achieve this. For example, the ratio between direct light and scattered light is used or the contribution of direct light and scattered light is predicted by training a deep learning model. This will not be elaborated in the embodiments of this application.

[0142] Direct light component refers to light rays that strike the object's surface directly from a virtual light source and are reflected or transmitted. This means light that reaches the object's surface directly without being blocked or scattered by other objects. It typically includes light from the main light source (such as sunlight or lamplight) and is responsible for creating strong reflective effects such as highlights and bright spots on the object's surface. Scattered light component, on the other hand, refers to the phenomenon of random reflections of light on the object's surface due to its roughness or material properties. Scattered light propagates uniformly and is not directly contributed by a specific light source. It primarily creates a dim, soft lighting effect on the object's surface, increasing the object's overall brightness.

[0143] In practice, after obtaining the direct light component and the scattered light component, the process of generating a soft shadow mask corresponding to the image to be rendered based on point cloud data, shadow mask, direct light component, and scattered light component can be as follows: Based on the direct light component and the scattered light component, determine the potential shadow region in the image to be rendered; Based on the point cloud data, shadow mask, and potential shadow region in the image to be rendered, predict the shadow degree of each pixel in the image to be rendered to obtain the shadow degree value of each pixel; Based on the shadow degree value of each pixel, obtain the soft shadow mask corresponding to the image to be rendered.

[0144] It should be noted that a potential shadow area refers to an area that may be covered by shadow under light. In other words, it's an area that, given the position of the light source and the direction of light propagation, might be blocked by other objects or surfaces, resulting in a shadow effect. When light propagates from a light source to an object's surface, if the light is blocked by other objects, preventing it from directly reaching a certain area, then that area will likely have a potential shadow.

[0145] Thus, in the rendering process of soft shadow images, taking into account potential shadow areas can not only help simulate the lighting effects in the real world, making the rendering of objects more realistic and lifelike, but also accurately simulate the propagation path of light in the rendering process, and determine the appropriate shadow effect according to the geometry of the object and the position of the light source, which helps to enhance the realism and three-dimensionality of soft shadow rendered images, making the lighting effects more natural and lifelike.

[0146] In practical implementation, the process of determining potential shadow regions in an image to be rendered based on the direct light component and the scattered light component can be as follows: For each pixel in the image to be rendered, perform the following processing: Based on the direct light component and the scattered light component, obtain the first brightness value of the pixel in the direct light dimension corresponding to the direct light component and the second brightness value in the scattered light dimension corresponding to the scattered light component; Based on the first brightness value and the second brightness value, select a target pixel from the multiple pixels included in the image to be rendered; wherein the first brightness value of the target pixel is less than the first brightness value threshold, and the second brightness value of the target pixel is greater than the second brightness value threshold, and the second brightness value threshold is greater than the first brightness value threshold; The area where the target pixel is located in the image to be rendered is determined as the potential shadow region in the image to be rendered.

[0147] For example, if a pixel is darker in the direct light component (i.e., darker when illuminated by direct light) and brighter in the diffuse light component (i.e., brighter when illuminated by diffuse light), then this pixel is a potential shadow area, meaning that the pixel is the target pixel.

[0148] It should be noted that the first brightness value of a pixel in the direct light dimension corresponding to the direct light component refers to the brightness value of the pixel when it is illuminated by direct light, and the second brightness value of a pixel in the scattered light dimension corresponding to the scattered light component refers to the brightness value of the pixel when it is illuminated by scattered light; at the same time, the first brightness value threshold and the second brightness value threshold can be preset.

[0149] In practical implementation, the process of generating a soft shadow mask for the image to be rendered based on point cloud data, shadow mask, direct light component, and diffuse light component is similar to the process described above of generating a soft shadow mask for the image to be rendered based on the point cloud data and shadow mask, and the process of generating a soft shadow mask for the image to be rendered based on point cloud data, shadow mask, and the illumination direction of the virtual light source. That is, based on point cloud data, shadow mask, direct light component, and diffuse light component, the shadow intensity of each pixel in the image to be rendered is predicted to obtain the shadow intensity value of each pixel; based on the shadow intensity value of each pixel, the soft shadow mask for the image to be rendered is obtained. This embodiment of the application will not elaborate further on this aspect.

[0150] Step 104: Render the image to be rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

[0151] In practice, after obtaining the soft shadow mask, the image to be rendered can be rendered directly based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered. Alternatively, the image to be rendered can be restored to obtain a shadowless image. Then, based on the soft shadow mask and the shadowless image, the image to be rendered can be rendered to obtain the soft shadow image corresponding to the image to be rendered.

[0152] In practical implementation, after generating a shadow mask corresponding to the shadow region in the image to be rendered, the shadow region in the image to be rendered can be eliminated based on the shadow mask to obtain a shadowless image. Therefore, the process of rendering the image based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered can be achieved by fusing the soft shadow mask with the shadowless image to obtain a merged image.

[0153]

[0154] in, It is the input soft shadow mask. It is the input image without shadows, F f It is the fused feature representation, f is a fused network, and θ f These are the parameters of the fusion network.

[0155] Then, the fused image is rendered to obtain the soft shadow image corresponding to the image to be rendered, i.e.

[0156]

[0157] in, It is the output soft shadow image, F f It is the fused feature representation, r is an image generation network, and θ r These are the parameters of the image generation network.

[0158] It should be noted that the process of removing shadows from the shadow areas in the image to be rendered based on the shadow mask to obtain a shadow-free image can be implemented based on a model, such as a diffusion model, or through other methods. This application does not limit the implementation of this method.

[0159] In some embodiments, when the process of removing shadows in the shadow areas of the image to be rendered based on the shadow mask to obtain a shadowless image is implemented based on the model, and the process of restoring the shadow areas in the image to be rendered based on the shadow mask to obtain a shadowless image can be: determining the shadow areas in the image to be rendered based on the shadow mask; and performing at least one denoising process on the image to be rendered based on the shadow areas to obtain a shadowless image.

[0160] The process of performing at least one denoising process on the image to be rendered to obtain a shadowless image can be implemented based on a diffusion model. Specifically, a pre-trained diffusion model is obtained, and then the image to be rendered is denoised at least once based on the diffusion model to obtain a shadowless image.

[0161] It should be noted that the number of denoising processes is preset and associated with the diffusion model. The diffusion model can transform any image restoration task into a mapping problem from a Gaussian noise image to a target image. The core idea of ​​this model is to gradually diffuse the latent representation of the image into Gaussian noise, then use a reverse diffusion network to gradually recover the latent representation of the image from the Gaussian noise, and finally use a decoder to convert the latent representation into the target image.

[0162] In this way, by recovering the shadowless image from the shadow image, the shadowless image can be applied to the subsequent generation of the soft shadow image. Compared with generating the soft shadow image directly based on the shadow image, this not only improves the clarity and realism of the soft shadow image, but also enhances the rendering effect of the soft shadow image.

[0163] In some embodiments, for the process of performing at least one denoising process on the image to be rendered to obtain a shadow-free image, when performing image feature processing, an anchored stripe self-attention mechanism or a window self-attention mechanism can be used. In this way, not only can the long-distance dependencies and local detail information of the image be captured, but also the computational and memory efficiency can be maintained, or the local context information and global perception ability of the image can be captured, while reducing the computational and memory overhead; moreover, efficient and explicit image hierarchical structure modeling is achieved, improving the accuracy of the image feature processing process.

[0164] It should be noted that the process of using the anchored fringe self-attention mechanism involves first segmenting the image to be rendered into multiple fringe patterns, then performing a self-attention operation on each fringe, while using anchor points to constrain the information flow between the fringe patterns. Here, the fringe pattern represents the distribution of anchor points in the input sequence, i.e., the intervals and positions between anchor points; and the anchor points are pre-selected to guide the self-attention model to focus on specific locations. In this way, the model can more effectively capture long-distance dependencies in the input sequence and avoid the problem of excessive computational resource consumption due to excessively long sequences.

[0165] Furthermore, when using the window self-attention mechanism, the image can be segmented into multiple windows during image feature processing, and the self-attention mechanism can be applied to each window. This allows the model to focus more intently on key regions in the input data, thereby improving the quality and accuracy of the generated image.

[0166] Applying the above embodiments of this application, a shadow mask corresponding to the image to be rendered, including the shadow region, is first generated. Then, based on the point cloud data corresponding to the image to be rendered and the shadow mask, a soft shadow mask corresponding to the image to be rendered is generated. Finally, the image to be rendered is rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered. Thus, obtaining the soft shadow image by rendering the image using the generated soft shadow mask not only reduces performance overhead but also improves the efficiency of rendering the soft shadow image. Simultaneously, the point cloud data corresponding to the image to be rendered is utilized during the soft shadow image rendering process, improving the rendering effect of the soft shadow image.

[0167] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0168] When rendering soft shadow images, most related technologies reconstruct the shadow image using geometric samples and the shadow mask after generating the corresponding shadow mask. However, this method requires massive computation, such as using sufficient samples to design 3D and color, making it difficult to achieve real-time rendering of shadow reconstruction. It not only has extremely high performance overhead, but also has low efficiency when rendering soft shadow images. At the same time, the method of reconstructing the shadow image using only geometric samples and the shadow mask will also result in a rather harsh shadow effect in the generated soft shadow image, reducing the rendering effect when rendering soft shadow images.

[0169] Based on this, embodiments of this application provide a method for rendering soft shadow images, which can recover a shadowless image from a shadow image, and then generate a realistic soft shadow effect based on a 3D point cloud and the direction of the light source. The main innovations of this method are as follows:

[0170] First, the method uses a generative model based on a diffusion process, which transforms any image restoration task into a mapping problem from a Gaussian noise image to the target image, thereby achieving efficient extraction and reconstruction of the latent representation of the image.

[0171] Second, this method utilizes the cross-scale similarity and anisotropy features of images, and achieves efficient and explicit image hierarchy modeling through anchored stripe self-attention mechanism and window self-attention mechanism, thereby improving image quality and detail.

[0172] Third, this method uses a shadow image decomposition model to decompose the shadow image into direct light components and scattered light components, thereby realizing the detection and segmentation of shadow areas and providing a basis for subsequent soft shadow generation.

[0173] Fourth, this method uses a shadow generation module to generate a soft shadow mask based on the 3D point cloud and the direction of the light source, thereby simulating and rendering the soft shadow effect.

[0174] See Figure 8 , Figure 8 This is a flowchart illustrating the soft shadow image rendering method provided in this application embodiment, based on... Figure 8 The soft shadow image rendering method provided in this application embodiment is implemented through steps 801 to 806. Specifically, the soft shadow image rendering method provided in this application embodiment consists of four steps: shadow detection, 3D projection, diffusion generation, and soft shadow generation.

[0175] The shadow detection process uses a simple shadow detection algorithm based on color information to convert the image to the HSV color space, thereby obtaining the brightness value of each pixel in the image. Then, a maximum and minimum value filter is used to identify shadow areas and non-shadow areas, and the pixel value of the shadow area is set to 1, while the pixel value of the non-shadow area is set to 0, thus obtaining the shadow mask.

[0176] In practical implementation, the purpose of shadow detection is to identify shadow regions in an image based on its color information and generate a shadow mask. Here, a simple shadow detection algorithm based on a maximum-minimum filter is used. This algorithm has the following advantages: First, it does not require prior knowledge of the light source's position and direction, nor does it require image segmentation or classification; second, it can handle different lighting conditions and shadow types, including hard shadows, soft shadows, and self-shadows; third, it has low computational complexity, making it suitable for real-time applications. The basic idea of ​​this algorithm is that the brightness value of a shadow region is usually lower than that of the surrounding non-shadow regions, so shadow regions can be identified by comparing the local maximum and minimum values ​​of the image. Specifically, the algorithm includes the following steps: First, the image is converted to the HSV color space to obtain the brightness value of each pixel in the image; then, the kernel size k of the maximum and minimum value filter is defined, which is generally an odd number, such as k = 15; then, as shown in formulas (2), (3), and (4), the maximum value filter is used to dilate the brightness channel to obtain the local maximum matrix M; then, as shown in formulas (5), (6), and (7), the minimum value filter is used to erode the brightness channel to obtain the local minimum matrix m; finally, as shown in formula (8), the shadow mask is calculated.

[0177] The 3D projection process involves using deep learning-based methods to estimate the depth map of the scene from the 2D image. Then, based on the depth map and camera parameters, each pixel in the 2D image is projected onto its corresponding pixel in 3D space. This yields a point cloud representation of the 3D scene.

[0178] In practice, the purpose of 3D projection is to map each pixel in the 2D image (the image to be rendered) to a point in 3D space based on the 2D image and camera parameters, thus obtaining a point cloud representation (point cloud data) of the 3D scene. This is a crucial step because it converts information from the 3D image into information in 3D space, providing a foundation for subsequent operations such as shadow generation and rendering. Here, a deep learning-based method is used to estimate the depth map of the scene from the 2D image, and then use projection transformation to convert the 2D coordinates to 3D coordinates. Specifically, the process includes the following steps: First, depth map estimation. Specifically, a depth estimation network is used, taking the 2D image as input and outputting the depth value of each pixel. The depth estimation network is a deep convolutional neural network that can learn the depth information of the scene from the 2D image without requiring any other supervision signals or prior knowledge. Different architectures and loss functions can be used for the depth estimation network, such as U-Net, ResNet, and BerHu. The size of the depth map is the same as the size of the 2D image, i.e., H×W, where H is the height of the image and W is the width of the image. Each pixel value D(i,j) in the depth map represents the distance from the pixel in the i-th row and j-th column of the image to the camera, in meters, as shown in formula (9).

[0179] Second, two-dimensional coordinate normalization. Specifically, to facilitate subsequent projection transformation, the pixel coordinates (i, j) in the two-dimensional image need to be converted into normalized homogeneous coordinates (x, y, 1), as shown in formulas (11) and (12); where the range of x and y is [-1, 1]. In this way, the influence of image size and resolution on projection transformation can be eliminated, so that projection transformation depends only on the camera's intrinsic parameter matrix.

[0180] Third, the calculation of three-dimensional coordinates. Specifically, as shown in formula (10), based on the camera's intrinsic parameter matrix K, the two-dimensional coordinates (x, y, 1) and the depth value D(i, j) can be converted into three-dimensional coordinates (X, Y, Z, 1), where X, Y, and Z represent the coordinates in the horizontal, vertical, and depth directions in three-dimensional space, respectively, in meters. The camera's intrinsic parameter matrix K is a 3×3 matrix containing the camera's focal length, principal point, and distortion parameters, which can generally be obtained from the camera calibration process.

[0181] Fourth, point cloud generation. Specifically, the 3D coordinates (X, Y, Z, 1) are stored as an N×4 matrix, where N = H×W is the total number of pixels in the image. This yields a point cloud representation of the 3D scene, which can be used for subsequent operations such as shadow generation and rendering. Point clouds are a commonly used 3D data structure; they can represent arbitrary shapes and objects in 3D space and can be used for various processing and analysis, such as plane fitting, surface reconstruction, and feature extraction.

[0182] The diffusion generation process utilizes a latent spatial diffusion model to recover a shadowless image from a shadowed image. This method leverages the cross-scale similarity and anisotropy features of images, employing anchored stripe self-attention and window self-attention mechanisms to achieve efficient and explicit image hierarchical structure modeling. This approach can process large-size real-world images and has achieved excellent results on various image restoration tasks.

[0183] In practical implementation, the purpose of diffusion generation is to recover a shadow-free image from a shadowed image, that is, to remove the shadow effect from the image and make the image clearer and more natural. Here, a method based on the latent spatial diffusion model is used. This method utilizes the cross-scale similarity and anisotropy features of images and achieves efficient and explicit image hierarchical structure modeling through anchored stripe self-attention mechanism and window self-attention mechanism. This method can handle large-size real images and has achieved excellent results on various image restoration tasks. Specifically, the latent spatial diffusion model can transform any image restoration task into a mapping problem from a Gaussian noise image to a target image. The core idea of ​​this model is to progressively diffuse the latent representation of the image into Gaussian noise, and then use a back-diffusion network to progressively recover the latent representation of the image from the Gaussian noise.

[0184]

[0185] Where z0 is the latent representation of the image, zt is the diffusion state at step t, βt is the diffusion coefficient at step t, and ∈t is the noise term at step t.

[0186] Finally, a decoder transforms the latent representation into the target image, i.e.

[0187]

[0188] in, φ is the output image without shadows, g is a decoder network, and φ is the parameter of the decoder network.

[0189] It should be noted that the backdiffusion network can be a deep convolutional neural network. The core idea of ​​this network is to use a U-Net structure with residual connections to extract and fuse features for each diffusion state, and then use a prediction module to predict the residual of the next diffusion state based on the current diffusion state and the noise term, thereby realizing the backdiffusion process.

[0190] In practical implementation, the anchored stripe self-attention mechanism is used. This mechanism is a self-attention-based image feature extraction and fusion method that can capture long-range dependencies and local details of an image while maintaining computational and memory efficiency. The core idea of ​​this mechanism is to segment the image into multiple stripes, then perform a self-attention operation on each stripe, while using anchor points to constrain the information flow between stripes, thereby achieving cross-stripe feature fusion.

[0191] In practical implementation, the window self-attention mechanism is used. This mechanism is a self-attention-based image feature extraction and fusion method that can capture local contextual information and global perception capabilities of an image while reducing computational and memory overhead. The core idea of ​​this mechanism is to segment the image into multiple windows, then perform a self-attention operation on each window, and use a learnable positional encoding to enhance the feature representation within the window, thereby achieving feature fusion within the window.

[0192] The soft shadow generation process utilizes a neural network-based method to generate realistic soft shadow effects from a shadowless image, a shadow mask, and a 3D point cloud. This method employs a shadow image decomposition model to break down the shadow image into direct light and diffuse light components. Then, a shadow generation module generates a soft shadow mask based on the 3D point cloud and the light source direction. Finally, a shadow fusion module fuses the soft shadow mask with the shadowless image to obtain the final soft shadow image.

[0193] In practical implementation, the goal of soft shadow generation is to produce realistic soft shadow effects based on a shadowless image, a shadow mask, and a 3D point cloud. Here, a neural network-based method is used. This method employs a shadow image decomposition model to decompose the shadow image into direct light and diffuse light components. Then, a shadow generation module generates a soft shadow mask based on the 3D point cloud and the light source direction. Finally, a shadow fusion module fuses the soft shadow mask with the shadowless image to obtain the final soft shadow image.

[0194] Among them, the shadow image decomposition model is a deep convolutional neural network used to decompose a shadow image into direct light components and scattered light components. The core idea of ​​this model is to use an encoder-decoder structure to extract and reconstruct features from the shadow image, while simultaneously detecting and segmenting shadow regions based on shadow masks, thereby achieving the decomposition of the shadow image.

[0195] The shadow generation module is a deep convolutional neural network used to generate soft shadow masks based on 3D point clouds and light source orientation. The core idea is to use a point cloud encoder to extract features from the 3D point cloud, and then use a shadow predictor to predict the shadow intensity of each point based on the point cloud features and light source orientation, thereby generating the soft shadow mask.

[0196] For the shadow fusion module, this module is a deep convolutional neural network used to fuse the soft shadow mask and the shadowless image to obtain the final soft shadow image. The core idea of ​​this module is to use a fusion network to extract and fuse features from the soft shadow mask and the shadowless image, and then use a reconstruction network (image generation network) to reconstruct and refine the fused features, thereby generating the soft shadow image, as shown in formulas (16) and (17).

[0197] Applying the above embodiments of this application, a shadow mask corresponding to the image to be rendered, including the shadow region, is first generated. Then, based on the point cloud data corresponding to the image to be rendered and the shadow mask, a soft shadow mask corresponding to the image to be rendered is generated. Finally, the image to be rendered is rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered. Thus, obtaining the soft shadow image by rendering the image using the generated soft shadow mask not only reduces performance overhead but also improves the efficiency of rendering the soft shadow image. Simultaneously, the point cloud data corresponding to the image to be rendered is utilized during the soft shadow image rendering process, improving the rendering effect of the soft shadow image.

[0198] The following description continues to illustrate the exemplary structure of the soft shadow image rendering apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module in the rendering device 455 storing the soft shadow image in the memory 450 may include:

[0199] The first acquisition module 4551 is used to acquire an image to be rendered, wherein the image to be rendered includes a shadow area;

[0200] The generation module 4552 is used to generate a shadow mask corresponding to the image to be rendered based on the shadow region in the image to be rendered;

[0201] The second acquisition module 4553 is used to acquire point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask.

[0202] The rendering module 4554 is used to render the image to be rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

[0203] In some embodiments, the generation module 4552 is further configured to obtain the brightness value of each pixel in the image to be rendered; and for each pixel in the image to be rendered, perform the following processing: dilate the pixel to obtain the local maximum brightness value of the pixel, and erode the pixel to obtain the local minimum brightness value of the pixel; determine the mask value of the pixel based on the local maximum brightness value and the local minimum brightness value; and obtain the shadow mask corresponding to the image to be rendered based on the mask value of each pixel.

[0204] In some embodiments, the apparatus further includes a third acquisition module, which is configured to acquire a target neighborhood range of the pixel, the target neighborhood range including at least one neighboring pixel adjacent to the pixel; the generation module 4552 is further configured to select the maximum brightness value from the brightness values ​​of each of the neighboring pixels and use the maximum brightness value as the local maximum brightness value of the pixel; and select the minimum brightness value from the brightness values ​​of each of the neighboring pixels and use the minimum brightness value as the local minimum brightness value of the pixel.

[0205] In some embodiments, the generation module 4552 is further configured to obtain the ratio of the minimum brightness value and the local maximum brightness value of the pixel; when the ratio is less than a target ratio, a first constant is determined as the mask value of the pixel; when the ratio is greater than or equal to the target ratio, a second constant is determined as the mask value of the pixel; wherein the first constant is different from the second constant.

[0206] In some embodiments, the second acquisition module 4553 is further configured to acquire the depth image of the image to be rendered and the coordinates of each pixel in the image to be rendered; based on the depth image, convert the coordinates of each pixel in the image to be rendered to obtain the three-dimensional coordinates of each pixel; and based on the three-dimensional coordinates of each pixel in the image to be rendered, obtain the point cloud data corresponding to the image to be rendered.

[0207] In some embodiments, the apparatus further includes an elimination module, which is configured to perform shadow elimination on the shadow area in the image to be rendered based on the shadow mask to obtain a shadowless image; the rendering module 4554 is further configured to fuse the soft shadow mask with the shadowless image to obtain a fused image; and to render the fused image to obtain a soft shadow image corresponding to the image to be rendered.

[0208] In some embodiments, the elimination module is further configured to determine the shadow region in the image to be rendered based on the shadow mask; and to perform at least one denoising process on the image to be rendered based on the shadow region to obtain the shadow-free image.

[0209] In some embodiments, the generation module 4552 is further configured to predict the shadow level of each pixel in the image to be rendered based on the point cloud data and the shadow mask, and obtain the shadow level value of each pixel; and obtain the soft shadow mask corresponding to the image to be rendered based on the shadow level value of each pixel.

[0210] In some embodiments, the apparatus further includes a first parsing module, which is configured to parse the image to be rendered to obtain the illumination direction of the virtual light source corresponding to the image to be rendered; the generation module 4552 is further configured to generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data, the shadow mask, and the illumination direction of the virtual light source.

[0211] In some embodiments, the apparatus further includes a second parsing module, which is used to parse the image to be rendered to obtain the direct light component and the scattered light component corresponding to the image to be rendered; the generation module 4552 is further used to generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data, the shadow mask, the direct light component and the scattered light component.

[0212] In some embodiments, the generation module 4552 is further configured to: determine potential shadow regions in the image to be rendered based on the direct light component and the scattered light component; predict the shadow degree of each pixel in the image to be rendered based on the point cloud data, the shadow mask, and the potential shadow regions in the image to be rendered, to obtain the shadow degree value of each pixel; and obtain the soft shadow mask corresponding to the image to be rendered based on the shadow degree value of each pixel.

[0213] In some embodiments, the generation module 4552 is further configured to perform the following processing for each pixel in the image to be rendered: based on the direct light component and the scattered light component, obtain a first brightness value of the pixel in the direct light dimension corresponding to the direct light component and a second brightness value in the scattered light dimension corresponding to the scattered light component; based on the first brightness value and the second brightness value, select a target pixel from the plurality of pixels included in the image to be rendered; wherein the first brightness value of the target pixel is less than a first brightness value threshold, and the second brightness value of the target pixel is greater than a second brightness value threshold, and the second brightness value threshold is greater than the first brightness value threshold; determine the area where the target pixel is located in the image to be rendered as the potential shadow area in the image to be rendered.

[0214] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the soft shadow image rendering method described above in this application embodiment, for example, as... Figure 3 The rendering method for soft shadow images is shown.

[0215] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the soft shadow image rendering method provided in this application. For example, ... Figure 3 The rendering method for soft shadow images is shown.

[0216] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or CD-ROM, etc.; or it may be a device that includes one or any combination of the above-mentioned memories.

[0217] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0218] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0219] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0220] It should be noted that in this application embodiment, data such as the image to be rendered is involved. When this application embodiment is applied to a specific product or technology, the corresponding license or consent needs to be obtained, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0221] In summary, the embodiments of this application have the following beneficial effects:

[0222] (1) The soft shadow image is obtained by rendering the image through the soft shadow mask corresponding to the generated image to be rendered. This not only reduces the performance overhead, but also improves the efficiency of rendering the soft shadow image. At the same time, the point cloud data corresponding to the image to be rendered is also used in the soft shadow image rendering process, which improves the rendering effect of the soft shadow image.

[0223] (2) The shadow mask is determined based on the local minimum brightness value and local maximum brightness value of the pixel, which helps to distinguish the brightness change area in the image to be rendered, thereby making it easier to distinguish the shadow area and the non-shadow area in the image to be rendered, and thus improving the rendering effect of the soft shadow image in the subsequent process.

[0224] (3) By normalizing the coordinates, the influence of image size and resolution on coordinate transformation can be eliminated, so that coordinate transformation depends only on the camera's intrinsic parameter matrix.

[0225] (4) By combining the direction of the virtual light emitted by the virtual light source corresponding to the image to be rendered, a soft shadow mask can be generated. This not only improves the accuracy of the generated soft shadow mask, but also improves the rendering effect of the subsequent soft shadow image. In other words, a realistic soft shadow effect can be generated, thereby enhancing the light and shadow effect and atmosphere of the soft shadow image.

[0226] (5) In the rendering process of soft shadow images, considering the potential shadow area can not only help simulate the lighting effect in the real world, making the rendering of objects more realistic and lifelike, but also accurately simulate the propagation path of light in the rendering, and determine the appropriate shadow effect according to the geometry of the object and the position of the light source, which helps to enhance the realism and three-dimensionality of the soft shadow rendering image, making the lighting effect more natural and lifelike.

[0227] (6) By recovering the shadowless image from the shadow image, the shadowless image is applied to the subsequent soft shadow image generation process. Compared with generating the soft shadow image directly based on the shadow image, this not only improves the clarity and realism of the soft shadow image, but also improves the rendering effect of the soft shadow image.

[0228] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for rendering soft shadow images, characterized in that, The method includes: Obtain an image to be rendered, which includes a shadow area; Based on the shadow region in the image to be rendered, generate a shadow mask corresponding to the image to be rendered; Obtain the point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask; The image to be rendered is rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

2. The method as described in claim 1, characterized in that, The step of generating a shadow mask corresponding to the image to be rendered based on the shadow region in the image to be rendered includes: Obtain the brightness value of each pixel in the image to be rendered; For each pixel in the image to be rendered, the following processing is performed: the pixel is dilated to obtain the local maximum brightness value, and the pixel is eroded to obtain the local minimum brightness value; based on the local maximum brightness value and the local minimum brightness value, the mask value of the pixel is determined. Based on the mask value of each pixel, the shadow mask corresponding to the image to be rendered is obtained.

3. The method as described in claim 2, characterized in that, Before performing dilation processing on the pixel to obtain the local maximum brightness value of the pixel, the method further includes: Obtain the target neighborhood range of the pixel, wherein the target neighborhood range includes at least one neighboring pixel adjacent to the pixel; The step of dilating the pixel to obtain the local maximum brightness value of the pixel includes: Select the maximum brightness value from the brightness values ​​of each of the adjacent pixels, and use the maximum brightness value as the local maximum brightness value of the pixel; The step of eroding the pixel to obtain the local minimum brightness value of the pixel includes: The minimum brightness value is selected from the brightness values ​​of each of the adjacent pixels, and the minimum brightness value is used as the local minimum brightness value of the pixel.

4. The method as described in claim 2 or 3, characterized in that, Determining the mask value of the pixel based on the local maximum brightness value and the local minimum brightness value includes: Obtain the ratio of the minimum brightness value to the local maximum brightness value of the pixel; When the ratio is less than the target ratio, the first constant is determined as the mask value of the pixel. When the ratio is greater than or equal to the target ratio, the second constant is determined as the mask value of the pixel. The first constant is different from the second constant.

5. The method according to any one of claims 1 to 4, characterized in that, The step of obtaining the point cloud data corresponding to the image to be rendered includes: Obtain the depth image of the image to be rendered, and the coordinates of each pixel in the image to be rendered; Based on the depth image, the coordinates of each pixel in the image to be rendered are transformed to obtain the three-dimensional coordinates of each pixel. Based on the three-dimensional coordinates of each pixel in the image to be rendered, the point cloud data corresponding to the image to be rendered is obtained.

6. The method according to any one of claims 1 to 5, characterized in that, After generating a shadow mask corresponding to the image to be rendered based on the shadow region in the image to be rendered, the method further includes: Based on the shadow mask, shadow removal is performed on the shadow areas in the image to be rendered to obtain a shadow-free image; The step of rendering the image to be rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered includes: The soft shadow mask is fused with the shadowless image to obtain a fused image; The fused image is rendered to obtain a soft shadow image corresponding to the image to be rendered.

7. The method as described in claim 6, characterized in that, The step of restoring the shadow region in the image to be rendered based on the shadow mask to obtain a shadow-free image includes: Based on the shadow mask, the shadow region in the image to be rendered is determined; Based on the shadow region, the image to be rendered is subjected to at least one denoising process to obtain the shadow-free image.

8. The method according to any one of claims 1 to 7, characterized in that, The step of generating a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask includes: Based on the point cloud data and the shadow mask, the shadow intensity of each pixel in the image to be rendered is predicted to obtain the shadow intensity value of each pixel. Based on the shadow intensity value of each pixel, the soft shadow mask corresponding to the image to be rendered is obtained.

9. The method according to any one of claims 1 to 8, characterized in that, After obtaining the image to be rendered, the method further includes: The image to be rendered is analyzed to obtain the illumination direction of the virtual light source corresponding to the image to be rendered; The step of generating a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask includes: Based on the point cloud data, the shadow mask, and the illumination direction of the virtual light source, a soft shadow mask corresponding to the image to be rendered is generated.

10. The method according to any one of claims 1 to 9, characterized in that, After obtaining the image to be rendered, the method further includes: The image to be rendered is analyzed to obtain the direct light component and the scattered light component corresponding to the image to be rendered; The step of generating a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask includes: Based on the point cloud data, the shadow mask, the direct light component, and the scattered light component, a soft shadow mask corresponding to the image to be rendered is generated.

11. The method as described in claim 10, characterized in that, The step of generating a soft shadow mask corresponding to the image to be rendered based on the point cloud data, the shadow mask, the direct light component, and the scattered light component includes: Based on the direct light component and the scattered light component, the potential shadow regions in the image to be rendered are determined; Based on the point cloud data, the shadow mask, and the potential shadow regions in the image to be rendered, the shadow intensity of each pixel in the image to be rendered is predicted to obtain the shadow intensity value of each pixel. Based on the shadow intensity value of each pixel, the soft shadow mask corresponding to the image to be rendered is obtained.

12. The method as described in claim 11, characterized in that, The step of determining potential shadow regions in the image to be rendered based on the direct light component and the scattered light component includes: For each pixel in the image to be rendered, the following processing is performed: Based on the direct light component and the scattered light component, the first brightness value of the pixel in the direct light dimension corresponding to the direct light component and the second brightness value in the scattered light dimension corresponding to the scattered light component are obtained respectively. Based on the first brightness value and the second brightness value, a target pixel is selected from the plurality of pixels included in the image to be rendered; Wherein, the first brightness value of the target pixel is less than the first brightness value threshold, and the second brightness value of the target pixel is greater than the second brightness value threshold, and the second brightness value threshold is greater than the first brightness value threshold; The region where the target pixel is located in the image to be rendered is determined as the potential shadow region in the image to be rendered.

13. A rendering apparatus for soft shadow images, characterized in that, The device includes: The first acquisition module is used to acquire an image to be rendered, wherein the image to be rendered includes a shadow area; The generation module is used to generate a shadow mask corresponding to the image to be rendered based on the shadow region in the image to be rendered; The second acquisition module is used to acquire point cloud data corresponding to the image to be rendered, and generate a soft shadow mask corresponding to the image to be rendered based on the point cloud data and the shadow mask. The rendering module is used to render the image to be rendered based on the soft shadow mask to obtain the soft shadow image corresponding to the image to be rendered.

14. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the soft shadow image rendering method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the soft shadow image rendering method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by the processor, the rendering method for soft shadow images according to any one of claims 1 to 12 is implemented.

Images