Image processing method and apparatus, and device, medium and program product

Abstract

Disclosed in the embodiments of the present application are an image processing method and apparatus, and a device and a storage medium. The method comprises: acquiring an i-th frame of cascaded shadow map; performing first reprojection processing on M shadow maps in the i-th frame of cascaded shadow map, so as to obtain M first maps; acquiring depth buffer information of pixel points in an (i+1)-th picture frame, and performing second reprojection processing on the pixel points in the (i+1)-th picture frame on the basis of the depth buffer information, so as to obtain M second maps; and updating the M shadow maps on the basis of the M first maps and the M second maps, so as to obtain an (i+1)-th frame of cascaded shadow map. It can be seen that by means of reprojection results of M shadow maps in an i-th frame of cascaded shadow map and reprojection results of pixel points in an (i+1)-th picture frame, the M shadow maps are updated to obtain an (i+1)-th frame of cascaded shadow map, such that computing resources required for updating a cascaded shadow map can be reduced.

Description

Image processing method, device, equipment, medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 14, 2023, with application number 2023107031315 and application name “A method, device, equipment and storage medium for image processing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to image processing. Background Art

[0003] With the advancement of scientific and technological research, image rendering technology has developed rapidly. Image rendering technology can restore the target scene by rendering the pixels in the image; for example, in the process of presenting a picture frame, the pixels in the picture frame can be rendered using the cascaded shadow map corresponding to the picture frame to obtain the rendered picture frame (i.e., the picture frame used for presentation). In practice, it has been found that when the target scene is more complex (such as the presence of shadows in the target scene), updating the cascaded shadow map requires a large amount of computing resources.

[0004] Summary of the Invention

[0005] Embodiments of the present application provide an image processing method, apparatus, device, and computer-readable storage medium that can reduce the computing resources required to update linked shadow maps.

[0006] In one aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] Get the cascaded shadow map of the i-th frame, the cascaded shadow map of the i-th frame is used to render the i-th frame, where i is a positive integer;

[0008] Reproject the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer;

[0009] Obtaining depth buffer information of each pixel in the (i+1)th frame, and performing a second reprojection process on the pixel in the (i+1)th frame based on the depth buffer information to obtain M second maps;

[0010] Based on the M first maps and the M second maps, the M shadow maps are updated to obtain an i+1-th frame cascaded shadow map; the i+1-th frame cascaded shadow map is used to render the i+1-th picture frame.

[0011] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0012] An acquisition unit, configured to acquire an i-th frame cascaded shadow map, wherein the i-th frame cascaded shadow map is used to render an i-th frame, where i is a positive integer;

[0013] a processing unit, configured to reproject M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer;

[0014] The acquisition unit is further used to obtain the depth buffer information of each pixel in the (i+1)th picture frame;

[0015] The processing unit is further configured to perform a second reprojection process on the pixel points in the (i+1)th picture frame based on the depth buffer information to obtain M second maps;

[0016] The method is used to update the M shadow maps based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map; the i+1-th frame cascaded shadow map is used to render the i+1-th picture frame.

[0017] Accordingly, the present application provides a computer device, comprising:

[0018] a memory, wherein a computer program is stored in the memory;

[0019] The processor is used to load a computer program to implement the above-mentioned image processing method.

[0020] Accordingly, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned image processing method.

[0021] Accordingly, the present application provides an embodiment of the present application which provides a computer program product including a computer program, which, when executed on a computer, enables the computer to execute the above method.

[0022] In an embodiment of the present application, a cascaded shadow map of the i-th frame is obtained, a first reprojection process is performed on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, depth buffer information of each pixel point in the i+1-th picture frame is obtained, and a second reprojection process is performed on the pixels in the i+1-th picture frame based on the depth buffer information to obtain M second maps, and the M shadow maps are updated based on the M first maps and the M second maps to obtain the cascaded shadow map of the i+1-th frame. It can be seen that by updating the M shadow maps using the reprojection results of the M shadow maps in the cascaded shadow map of the i-th frame and the reprojection results of each pixel point in the i+1-th picture frame to obtain the cascaded shadow map of the i+1-th frame, the computing resources required for updating the cascaded shadow maps can be reduced, and better shadow rendering effects can be obtained, especially for scenes where the light source changes in the picture frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a diagram of an image processing scenario provided by an embodiment of the present application;

[0024] FIG2 is a flow chart of an image processing method provided in an embodiment of the present application;

[0025] FIG3 is a flowchart of another image processing method provided by an embodiment of the present application;

[0026] FIG4a is a schematic diagram of a spatial position conversion relationship provided in an embodiment of the present application;

[0027] FIG4 b is a schematic diagram of adjacent pixels of a target pixel provided by an embodiment of the present application;

[0028] FIG4c is a schematic diagram of neighboring pixels of another target pixel provided by an embodiment of the present application;

[0029] FIG5 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0030] FIG6 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] This application involves technologies related to image rendering. The following is a brief introduction to the relevant terms involved:

[0033] Shadow Map: refers to a depth map of a scene drawn at the position of the light source in the scene in the direction of the light source; the shadow map can be used to render shadows in the scene.

[0034] Cascade Shadow Map: A technique for rendering shadow effects that improves the shadow accuracy of the scene's rendered image by drawing one or more shadow maps on the scene's camera viewpoint (i.e., the current observation viewpoint). Specifically, if the frustum of the acquisition device associated with the camera viewpoint is divided into N spatial blocks based on the distance from the acquisition device, then N shadow maps need to be drawn on the camera viewpoint, with each shadow map corresponding to a spatial block of the frustum, where N is a positive integer.

[0035] In response to the above-mentioned image rendering technology, an embodiment of the present application provides an image processing method that can reduce the computing resources required to update the linked shadow map. Figure 1 is an image processing scenario diagram provided by an embodiment of the present application. As shown in Figure 1, the image processing scenario provided by the present application includes a computer device 101. The image processing solution provided by the present application can be executed by the computer device 101. The computer device 101 can be a terminal device or a server. Among them, the terminal device can include but is not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, portable personal computers, mobile Internet devices (Mobile Internet Devices, referred to as MID), vehicle terminals, smart home appliances, wearable devices, etc., and the embodiment of the present application does not limit this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and basic cloud computing services such as big data and artificial intelligence platforms. The embodiment of the present application does not limit this.

[0036] It should be noted that the number of computer devices in Figure 1 is for example only and does not constitute an actual limitation of this application; for example, Figure 1 may also include computer device 102 (such as for sending the cascaded shadow map of the i-th frame to computer device 101), and computer device 101 and computer device 102 can be connected via wired or wireless means, and this application does not impose any restrictions on this.

[0037] In specific implementation, the general principle of this image processing solution is as follows:

[0038] (1) The computer device 101 obtains the cascaded shadow map of the i-th frame, and the cascaded shadow map of the i-th frame is used to render the i-th picture frame, where i is a positive integer. In one embodiment, the computer device 101 divides the viewing cone of the acquisition device corresponding to the i-th picture frame (camera perspective) into N spatial blocks (each spatial block has a different depth range and does not overlap with each other) based on the scene depth corresponding to the i-th picture frame, where N is a positive integer; and generates a bounding box for each spatial block under the light source perspective of the scene. After obtaining the bounding box of each spatial block, the computer device 101 generates a shadow map of the spatial block based on the bounding box of each spatial block, and adds the shadow map of each spatial block to the map set to obtain the cascaded shadow map of the i-th frame. It can be understood that in the above embodiment, the cascaded shadow map of the i-th frame includes N shadow maps.

[0039] (2) The computer device 101 performs a first reprojection process on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer. Any shadow map in the cascaded shadow map of the i-th frame is represented as the j-th shadow map in the cascaded shadow map of the i-th frame. For the j-th shadow map in the cascaded shadow map of the i-th frame, the first reprojection process means: using the shadow projection inverse matrix of the j-th shadow map in the cascaded shadow map of the i-th frame, the position (such as coordinate) of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space (UV space) is converted to a position in the world space, and then using the shadow projection matrix of the i+1-th frame cascaded shadow map, the position of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the world space is converted to a position in the shadow space, so as to achieve the purpose of updating the position of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space. After obtaining the updated positions of each pixel in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space, the map to be updated is updated using the updated positions of each pixel in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space, thereby obtaining a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame. The map to be updated is the j-th shadow map in the cascaded shadow map of the i-th frame, or a preset shadow map (e.g., a blank map in which the depth values ​​of each pixel are preset values).

[0040] (3) The computer device 101 obtains the depth buffer information (Depth Buffer) of the i+1th frame, and performs a second reprojection process on the pixels in the i+1th frame based on the depth buffer information of the i+1th frame to obtain M second maps. The depth buffer information of the i+1th frame carries the depth value of each pixel in the i+1th frame at the camera perspective (observation perspective). The M first maps correspond one-to-one to the M second maps, and the corresponding first maps and second maps are used to update the same shadow map in the M shadow maps in the cascaded shadow map of the i-th frame.

[0041] The computer device 101 performs a second reprojection process on the pixels in the i+1th frame based on the depth buffer information of the i+1th frame. This process involves the computer device 101 determining the position of each pixel in the view space using the position of the pixel in the i+1th frame and the depth value of the pixel. The computer device 101 then converts the position of each pixel in the view space (corresponding to the observation angle of the i+1th frame) to a position in the world space using the view projection matrix (which can be used to indicate the conversion (projection) relationship between the position of a pixel in the view space and the position in the world space) of the acquisition device corresponding to the i+1th frame (the acquisition devices corresponding to different frames can be the same or different, and each acquisition device corresponds to a view projection matrix). After converting the view space positions of each pixel in the i+1th frame to positions in the world space, the computer device 101 converts the world space positions of each pixel in the i+1th frame to positions in the shadow space using the shadow projection matrix of the i+1th frame's cascaded shadow map. The to-be-updated map is then updated based on the positions of each pixel in the shadow space in the (i+1)th frame, resulting in a second map corresponding to the jth shadow map in the cascaded shadow maps of the i-th frame. The to-be-updated map is either the jth shadow map in the cascaded shadow maps of the i-th frame, or a preset shadow map (e.g., a blank map with preset depth values ​​for each pixel).

[0042] (4) The computer device updates the M shadow maps based on the M first maps and the M second maps to obtain the i+1 frame cascaded shadow map. The M first maps correspond one-to-one to the M second maps, and the j-th shadow map in the i+1 frame cascaded shadow map is obtained by updating the j-th shadow map in the i-frame cascaded shadow map based on the first map corresponding to the j-th shadow map in the i-frame cascaded shadow map and the second map corresponding to the j-th shadow map in the i-frame cascaded shadow map. The i+1 frame cascaded shadow map is used to render the i+1 frame. In one embodiment, the computer device 101 fuses each first map with the second map corresponding to the first map to obtain M fused maps, and filters the M fused maps to obtain M filtered fused maps. After obtaining the M filtered fused maps, the computer device 101 updates the M shadow maps of the cascaded shadow map of the i-th frame using the M filtered fused maps to obtain an updated cascaded shadow map of the (i+1)-th frame.

[0043] In an embodiment of the present application, a cascaded shadow map of the i-th frame is obtained, a first reprojection process is performed on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, the depth buffer information of each pixel in the i+1-th picture frame is obtained, and a second reprojection process is performed on the pixels in the i+1-th picture frame based on the depth buffer information to obtain M second maps, and the M shadow maps are updated based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map. It can be seen that by updating the M shadow maps using the reprojection results of the M shadow maps in the cascaded shadow map of the i-th frame and the reprojection results of the depth buffer information of the i+1-th picture frame to obtain the i+1-th frame cascaded shadow map, the computing resources required to update the cascaded shadow maps can be reduced.

[0044] Based on the above-mentioned image processing scheme, the embodiment of the present application proposes a more detailed image processing method. The image processing method proposed in the embodiment of the present application will be introduced in detail below with reference to the accompanying drawings.

[0045] Please refer to Figure 2, which is a flowchart of an image processing method provided in an embodiment of the present application. The image processing method can be executed by a computer device, which can specifically be a terminal device or a server. As shown in Figure 2, the image processing method may include the following steps S201-S204:

[0046] S201: Obtain the cascaded shadow map of the i-th frame.

[0047] The i-th frame cascade shadow map is used to render the i-th picture frame, where i is a positive integer. One picture frame corresponds to one frame of cascade shadow map. In one embodiment, the computer device divides the frustum of the acquisition device corresponding to the i-th picture frame (camera perspective) into N spatial blocks (each spatial block has a different depth range and does not overlap with each other) based on the scene depth corresponding to the i-th picture frame, where N is a positive integer; and generates a bounding box for each spatial block under the perspective of the light source of the scene. After obtaining the bounding box of each spatial block, the computer device generates a shadow map for the spatial block based on the bounding box of each spatial block, and adds the shadow map of each spatial block to the map set to obtain the i-th frame cascade shadow map. Each shadow map carries the depth value of each pixel in the shadow map. Cascade Shadow Mapping is an image rendering technique used to create shadow effects. It improves the shadow accuracy of the scene's rendered image by drawing one or more shadow maps (a depth map of the scene drawn from the scene's light source position toward the light source's direction) on the scene's camera perspective (i.e., the current observation perspective). In other words, the cascade shadow map corresponding to each frame is composed of one or more shadow maps.

[0048] S202 : Perform a first reprojection process on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps.

[0049] The j-th shadow map in the cascaded shadow map of the i-th frame is any one of the M shadow maps in the cascaded shadow map of the i-th frame; j and M are both positive integers less than or equal to N, and N is the number of spatial blocks after the view frustum of the acquisition device corresponding to the i-th frame is divided. For the j-th shadow map in the cascaded shadow map of the i-th frame, the first reprojection processing refers to: using the shadow projection inverse matrix of the j-th shadow map in the cascaded shadow map of the i-th frame, converting the position (such as coordinates) of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space (UV space) to a position in the world space, and then using the shadow projection matrix of the cascaded shadow map of the i+1-th frame, converting the position of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the world space to a position in the shadow space, so as to achieve the purpose of updating the position of each pixel point in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space.

[0050] Among them, the shadow projection matrix and the shadow projection inverse matrix are used to indicate the conversion relationship between the position of the same pixel point in the shadow space and the position of the pixel point in the world space. The shadow projection inverse matrix of the j-th shadow map in the cascaded shadow map of the i-th frame and the shadow projection matrix of the j-th shadow map in the cascaded shadow map of the i-th frame are inverse matrices of each other. Specifically, the shadow projection matrix of the j-th shadow map in the cascaded shadow map of the i-th frame is used to project the pixel points in the world space into the shadow space; the shadow projection inverse matrix of the j-th shadow map in the cascaded shadow map of the i-th frame is used to project the pixel points in the shadow space into the world space; similarly, the shadow projection matrix of the j-th shadow map in the cascaded shadow map of the i+1-th frame is used to project the pixel points in the world space into the shadow space.

[0051] After obtaining the updated positions of each pixel in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space, the map to be updated is updated using the updated positions of each pixel in the j-th shadow map in the cascaded shadow map of the i-th frame in the shadow space, thereby obtaining a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame. The map to be updated is the j-th shadow map in the cascaded shadow map of the i-th frame, or a preset shadow map (e.g., a blank map in which the depth values ​​of each pixel are preset values).

[0052] After the positions of the pixels in the jth shadow map of the cascaded shadow map of the i-th frame are updated, these pixels are positionally associated with the i+1th frame. When the updated shadow map is updated using the updated positions, the depth values ​​in the updated shadow map can be adjusted based on the depth values ​​of the pixels in the i-th frame, so that the depth values ​​in the i-th frame are accurately reflected in the first shadow map. The updating method may include replacing the depth values.

[0053] In one embodiment, each shadow map carries a depth value for each pixel in the shadow map. The computer device performs a first reprojection process on the M shadow maps in the cascaded shadow maps of the i-th frame to obtain M (M is a positive integer) first maps, including obtaining a shadow projection matrix for the j-th shadow map of the cascaded shadow maps of the i+1-th frame, and then determining, based on the depth values ​​of each pixel in the j-th shadow map of the cascaded shadow maps of the i-th frame, a position of each pixel in the j-th shadow map of the cascaded shadow maps of the i-th frame in the shadow space.

[0054] Furthermore, the computer device updates the position of each pixel in the j-th shadow map of the cascaded shadow map of the i+1-th frame in the shadow space through the shadow projection matrix of the j-th shadow map of the cascaded shadow map; and based on the updated position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space, updates the map to be updated to obtain a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame. The map to be updated may refer to the j-th shadow map of the cascaded shadow map of the i-th frame, or may refer to the original map, or other preset map; the depth value of each pixel in the original map is a preset value (such as 1, positive infinity, etc.).

[0055] According to the depth values ​​of the pixels carried in the shadow map, the position of each pixel to be processed in the shadow space can be accurately grasped during the first reprojection process, thereby improving the accuracy of the first map.

[0056] S203 , obtaining depth buffer information of each pixel in the (i+1)th picture frame, and performing a second reprojection process on the pixel in the (i+1)th picture frame based on the depth buffer information to obtain M second maps.

[0057] The depth buffer information of the i+1th frame carries (stores) the depth value of each pixel in the i+1th frame from the camera's perspective (observation perspective). The M first maps correspond one-to-one to the M second maps, and the corresponding first maps and second maps are used to update the same shadow map in the M shadow maps in the cascaded shadow map of the i-th frame. For example, assuming that the target first map is the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, and the target second map is the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, then the target first map and the target second map correspond to each other, and both the target first map and the target second map are used to update the j-th shadow map in the M shadow maps in the cascaded shadow map of the i-th frame.

[0058] In one embodiment, on the one hand, the computer device can determine the position (x, y, z) of each pixel in the view space based on the position (x, y) of each pixel in the i+1th frame and the depth value (z) of the pixel. On the other hand, the computer device obtains the view projection matrix (of the acquisition device) corresponding to the i+1th frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascade shadow map; wherein the acquisition devices corresponding to different frames can be the same or different, and each acquisition device corresponds to a view projection matrix, which can be used to indicate the conversion (projection) relationship between the position of the pixel in the view space and the position in the world space. For example, assuming that picture frame 1 corresponds to capture device 1, picture frame 2 corresponds to capture device 2, capture device 1 corresponds to view projection matrix 1, and capture device 2 corresponds to view projection matrix 2; then the view projection matrix corresponding to picture frame 1 is view projection matrix 1, and picture frame 2 corresponds to view projection matrix 2; view projection matrix 1 can be used to convert the position of the pixel points in picture frame 1 in the view space to the position of the pixel points in picture frame 1 in the world space; view projection matrix 2 can be used to convert the position of the pixel points in picture frame 2 in the view space to the position of the pixel points in picture frame 2 in the world space. The shadow projection matrix and the inverse shadow projection matrix are used to indicate the conversion relationship between the position of the same pixel point in the shadow space and the position of the pixel point in the world space; the shadow projection matrix of the jth shadow map in the cascaded shadow map of the i+1th frame is used to project the pixel points in the world space into the shadow space.

[0059] The computer device then converts the position of each pixel in the i+1th frame in the view space into a position in the shadow space through the view projection matrix (of the acquisition device) corresponding to the i+1th frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map. Furthermore, the computer device updates the map to be updated based on the position of each pixel in the i+1th frame in the shadow space, and obtains a second map corresponding to the jth shadow map in the i-th frame cascaded shadow map. The map to be updated may refer to the jth shadow map of the i-th frame cascaded shadow map, or may refer to the original map, or other preset maps; the depth values ​​of each pixel in the original map are all preset values ​​(such as 1, positive infinity, etc.).

[0060] Among them, by converting the positions of the view space and the shadow space, and updating the map to be updated based on the converted position, a second map is obtained. The second map can reflect the position of the shadow part in the i+1th frame. The update mainly updates the depth value of the pixel point in the map to be updated, and the updating method can include replacing the depth value.

[0061] S204 : Based on the M first maps and the M second maps, update the M shadow maps to obtain a cascaded shadow map of the (i+1)th frame.

[0062] The j-th shadow map in the cascaded shadow map of the i+1th frame is obtained by updating the j-th shadow map in the cascaded shadow map of the i-th frame based on the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame. The i+1-th frame cascaded shadow map is used to render the i+1-th frame.

[0063] In this step, the first map and the second map can effectively reflect the association between the shadow parts in the i-th frame and the i+1-th frame. When updating the M shadow maps, the depth values ​​of each pixel in the shadow map are mainly updated. For example, the depth values ​​in the shadow map are replaced according to the depth values ​​in the first map and the second map in units of pixels.

[0064] In one embodiment, M first tiles correspond one-to-one to M second tiles. The computer device fuses each first tile with the second tile corresponding to the first tile to obtain M fused tiles. The computer device then filters the M fused tiles to obtain M filtered fused tiles. After obtaining the M filtered fused tiles, the computer device updates the M shadow maps of the cascaded shadow map of the i-th frame using the M filtered fused tiles to obtain an updated cascaded shadow map of the i+1-th frame.

[0065] In another embodiment, the computer device updates the M shadow maps of the cascaded shadow map of the i-th frame using M first maps (the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is used to update the j-th shadow map), thereby obtaining a first updated result for the M shadow maps of the cascaded shadow map of the i-th frame. The computer device then further updates the first updated result for the M shadow maps of the cascaded shadow map of the i-th frame using M second maps (the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is used to update the first updated result for the j-th shadow map), thereby obtaining a second updated result for the M shadow maps of the i-th frame. After obtaining the second updated result for the M shadow maps of the cascaded shadow map of the i-th frame, the computer device filters the second updated result for the M shadow maps of the i-th frame to obtain an updated cascaded shadow map of the i+1-th frame. It is understandable that the update order of the shadow map by the first map and the second map can be swapped, and this application does not impose any limitation on this.

[0066] In an embodiment of the present application, a cascaded shadow map of the i-th frame is obtained, a first reprojection process is performed on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, depth buffer information of each pixel in the i+1-th frame is obtained, and a second reprojection process is performed on the pixels in the i+1-th frame based on the depth buffer information to obtain M second maps, and the M shadow maps are updated based on the M first maps and the M second maps to obtain a cascaded shadow map of the i+1-th frame. It can be seen that by updating the M shadow maps using the reprojection results of the M shadow maps in the cascaded shadow map of the i-th frame and the reprojection results of the depth buffer information of the i+1-th frame to obtain the cascaded shadow map of the i+1-th frame, the computing resources required for updating the cascaded shadow maps can be reduced; further, by rendering the i+1-th frame using the updated cascaded shadow map of the i+1-th frame, the rendering effect of the frame can be improved.

[0067] Please refer to Figure 3, which is a flowchart of another image processing method provided in an embodiment of the present application. The image processing method can be executed by a computer device, which can specifically be a terminal device or a server. As shown in Figure 3, the image processing method may include the following steps S301-S311:

[0068] S301: Obtain the cascaded shadow map of the i-th frame.

[0069] The specific implementation of step S301 can refer to the implementation of step S201 in Figure 2, and will not be repeated here.

[0070] S302 : Obtain the shadow projection matrix of the j-th shadow map of the (i+1)-th frame cascaded shadow map.

[0071] The j-th shadow map in the cascaded shadow map of the i+1th frame can be obtained by updating any one of the M shadow maps in the cascaded shadow map of the i-th frame. For example, assuming that the cascaded shadow map of the i-th frame includes 3 shadow maps (such as shadow map 1-shadow map 3), if the shadow maps that need to be updated in the cascaded shadow map of the i-th frame are shadow map 1 and shadow map 3 (i.e., M=2), then the j-th shadow map in the cascaded shadow map of the i+1th frame can be obtained by updating shadow map 1 in the cascaded shadow map of the i-th frame, or by updating shadow map 3 in the cascaded shadow map of the i-th frame.

[0072] In one embodiment, a computer device obtains a view matrix (of an acquisition device) corresponding to the i+1th frame and a projection matrix corresponding to the jth level in the i+1th frame cascaded shadow map, and performs product processing on the view matrix and the projection matrix to obtain a shadow projection matrix of the jth level shadow map in the i+1th frame cascaded shadow map. The view matrix (of an acquisition device) corresponding to the i+1th frame is obtained based on the position and orientation of the light source in the scene to which the object belongs at the i+1th moment, and the i+1th moment corresponds to the i+1th frame; the projection matrix corresponding to the jth level in the i+1th frame cascaded shadow map is obtained based on the bounding box corresponding to the jth level in the i+1th frame cascaded shadow map.

[0073] S303 : Determine the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space based on the depth value of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame.

[0074] The j-th shadow map in the cascaded shadow maps of the i-th frame can be any shadow map among the M shadow maps in the cascaded shadow maps of the i-th frame. For example, assuming that the cascaded shadow maps of the i-th frame include three shadow maps (such as shadow map 1-shadow map 3), where the shadow maps that need to be updated are shadow map 1 and shadow map 3, then the j-th shadow map in the cascaded shadow maps of the i-th frame can be shadow map 1 or shadow map 3.

[0075] In one embodiment, a computer device determines the position (which can be represented by three-dimensional space coordinates (x, y, z)) of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow space based on the position of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow map (determining the coordinates of each pixel point on the x-axis and y-axis in the view space), and the depth value of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame (determining the coordinate of each pixel point on the z-axis in the view space).

[0076] S304 , updating the position of each pixel point in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space by using the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame.

[0077] Each shadow map carries the depth value of each pixel in the shadow map. In one embodiment, the computer device obtains the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame (which can be obtained through the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i-th frame). The computer device performs a first conversion (projection) on the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame through the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame, and obtains the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the world space. That is, the first conversion refers to converting the position of the pixel in the shadow space into the position of the pixel in the world space through the shadow projection inverse matrix.

[0078] The above implementation can also be understood as that the computer device determines the conversion relationship between the positions of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow space and the positions of these pixel points in the world space through the shadow projection inverse matrix of the jth shadow map of the cascaded shadow map of the i-th frame, and then converts the position of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow space according to the conversion relationship to obtain the position of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the world space.

[0079] After obtaining the world space positions of each pixel in the jth shadow map of the cascaded shadow map of the i-th frame, the computer device performs a second transformation on the world space positions of each pixel in the jth shadow map of the cascaded shadow map of the i+1-th frame using the shadow projection matrix of the jth shadow map of the cascaded shadow map, thereby obtaining updated shadow space positions of each pixel in the jth shadow map of the cascaded shadow map of the i-th frame. In other words, the second transformation converts the world space positions of the pixels into shadow space positions of the pixels using the shadow projection matrix.

[0080] It can be understood that when the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i-th frame is different from the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame, the first conversion result and the second conversion result are different; wherein, the first conversion result is the position of the target pixel point in the shadow space obtained by converting (projecting) the position of the target pixel point in the world space based on the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i-th frame; the second conversion result is the position of the target pixel point in the shadow space obtained by converting (projecting) the position of the target pixel point in the world space based on the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame.

[0081] Through the shadow projection inverse matrix and shadow projection matrix of adjacent cascaded shadow maps, the shadow differences between adjacent cascaded shadow maps can be effectively captured, thereby improving the accuracy of the subsequent generation of the i+1th frame cascaded shadow map.

[0082] S305 , based on the updated positions of the pixels in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space, update the map to be updated to obtain a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame.

[0083] The map to be updated can be the j-th shadow map of the cascaded shadow map of the i-th frame, or it can be a preset shadow map (such as a shadow map whose scale is the same as the j-th shadow map of the cascaded shadow map of the i-th frame, and the depth value of each pixel is positive infinity). The first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame can be used to update the j-th shadow map in the cascaded shadow map of the i-th frame. It can be understood that if the map to be updated is the j-th shadow map of the cascaded shadow map of the i-th frame, then the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is the shadow map after the j-th shadow map in the cascaded shadow map of the i-th frame is updated.

[0084] In one embodiment, a computer device updates the map to be updated (such as through an InterlockedMin operation) based on the updated positions of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space, and a process for obtaining a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes: determining the map position and depth update value of the k-th pixel based on the updated position of the k-th pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space; specifically, the updated position of the k-th pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space can be represented by three-dimensional space coordinates (x, y, z); wherein (x, y) is used to indicate the map position of the k-th pixel (in the shadow space), and z is used to indicate the depth update value of the k-th pixel (in the shadow space). The map to be updated may refer to the jth shadow map of the cascaded shadow map of the i-th frame, or may refer to the original map, or other preset maps; the depth values ​​of each pixel in the original map are all preset values ​​(such as 1, positive infinity, etc.).

[0085] If the depth value of the target pixel in the map to be updated is greater than the depth update value of the k-th pixel, the computer device updates the depth value of the target pixel to the depth update value of the k-th pixel; correspondingly, if the depth value of the target pixel in the map to be updated is less than or equal to the depth update value of the k-th pixel, the computer device maintains the depth value of the target pixel unchanged. The position of the target pixel in the map to be updated matches the map position of the k-th pixel; and the k-th pixel in the j-th shadow map of the cascaded shadow map of the i-th frame is any pixel in the j-th shadow map of the cascaded shadow map of the i-th frame.

[0086] For example, assuming that the k-th pixel in the j-th shadow map of the i-th frame cascaded shadow map has a map position of (x1, y1) and a depth value of z1 before updating, and a map position of (x2, y2) and a depth value of z2 after updating, the pixel at (x2, y2) in the map to be updated is the target pixel, and the depth value of the target pixel is z3; if z2 is greater than or equal to z3, the depth value z3 of the target pixel remains unchanged; if z2 is less than z3, the depth value of the target pixel is updated to z2. By comparing the depth values ​​of the target pixel in the map to be updated with the k-th pixel that matches its position in the j-th shadow map, the pixels belonging to the shadow part can be quickly determined in the map to be updated, thereby improving the efficiency of generating the i+1-th frame cascaded shadow map.

[0087] S306: Obtain the view projection matrix corresponding to the (i+1)th picture frame.

[0088] The view projection matrix corresponding to the i+1th frame refers to the view projection matrix (of the acquisition device) corresponding to the i+1th frame. The view projection matrix (of the acquisition device) corresponding to the i+1th frame is used to convert the position of each pixel in the i+1th frame in view space (corresponding to the observation angle of the i+1th frame) to a position in world space. In other words, the view projection matrix is ​​used to indicate the conversion relationship between the position of the same pixel in view space and the position of the same pixel in world space.

[0089] Figure 4a is a schematic diagram of a spatial position conversion relationship provided by an embodiment of the present application. As shown in Figure 4a, the position of a pixel point in the view space can be converted to the position of the pixel point in the world space through the view projection matrix; correspondingly, the position of the pixel point in the world space can be converted to the position of the pixel point in the view space through the inverse matrix of the view projection matrix. In addition, the position of the pixel point in the world space can be converted to the position of the pixel point in the shadow space through the shadow projection matrix; correspondingly, the position of the pixel point in the shadow space can be converted to the position of the pixel point in the world space through the inverse matrix of the shadow projection matrix (and the shadow projection inverse matrix). Among them, the position of the pixel point in the view space can be obtained through the picture frame and the depth buffer information of the picture frame; the position of the pixel point in the shadow space can be obtained through the shadow map.

[0090] S307 : Determine the position of each pixel in the (i+1)th picture frame in the view space based on the position of each pixel in the (i+1)th picture frame and the depth buffer information.

[0091] In one embodiment, the computer device determines the x-axis coordinate and y-axis coordinate of the k-th pixel based on the position (x, y) of the k-th pixel in the i+1-th frame; and determines the z-axis coordinate (i.e., depth value) of the k-th pixel through the depth buffer information of the i+1-th frame (carrying the depth values ​​of each pixel in the i+1-th frame), thereby obtaining the position of the k-th pixel in the view space (represented by (x, y, z)).

[0092] S308. Convert the position of each pixel in the i+1th frame in the view space to the position in the shadow space using the view projection matrix corresponding to the i+1th frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map.

[0093] In one embodiment, the computer device performs a third transformation on the position of each pixel in the (i+1)th image frame in view space using the view projection matrix to obtain the position of each pixel in the (i+1)th image frame in world space. In other words, the third transformation converts the position of the pixel in view space into the position of the pixel in world space using the view projection matrix.

[0094] After obtaining the positions of each pixel in the i+1th frame in world space, the computer device performs a second transformation on the positions of each pixel in the i+1th frame in world space using the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map, thereby obtaining the positions of each pixel in the i+1th frame in shadow space. The above embodiment can also be understood as the view projection matrix determining the transformation relationship between the positions of each pixel in the i+1th frame in view space and the positions of these pixels in world space, thereby transforming the positions of each pixel in the i+1th frame in view space based on the transformation relationship to obtain the positions of each pixel in the i+1th frame in world space.

[0095] S309 : Based on the position of each pixel point in the shadow space in the (i+1)th picture frame, update the map to be updated to obtain a second map corresponding to the jth shadow map in the cascaded shadow map of the (i)th frame.

[0096] The map to be updated can be the j-th shadow map in the cascaded shadow maps of the i-th frame, or it can be a preset shadow map (such as a shadow map obtained by updating the j-th shadow map in the cascaded shadow map of the i-th frame using the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame). The second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame can be used to update the j-th shadow map in the cascaded shadow map of the i-th frame. It can be understood that if the map to be updated is the j-th shadow map in the cascaded shadow map of the i-th frame, then the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is the shadow map after updating the j-th shadow map in the cascaded shadow map of the i-th frame.

[0097] In one embodiment, a computer device updates a to-be-updated map based on the positions of each pixel in the i+1th frame in the shadow space (e.g., through an InterlockedMin operation) to obtain a second map corresponding to the jth shadow map in the i-th frame cascaded shadow map, including: determining the map position and depth value of the kth pixel based on the position of the kth pixel in the i+1th frame in the shadow space; specifically, the position of the kth pixel in the jth shadow map of the i-th frame cascaded shadow map in the shadow space can be represented by three-dimensional spatial coordinates (x, y, z); wherein (x, y) is used to indicate the map position of the kth pixel (in the shadow space), and z is used to indicate the depth value of the kth pixel (in the shadow space). The to-be-updated map can refer to the jth shadow map of the i-th frame cascaded shadow map, or can refer to the original map, or other preset maps; the depth values ​​of each pixel in the original map are preset values ​​(e.g., 1, positive infinity, etc.).

[0098] If the depth value of the target pixel in the map to be updated is greater than the depth update value of the k-th pixel, the computer device updates the depth value of the target pixel to the depth value of the k-th pixel; correspondingly, if the depth value of the target pixel in the map to be updated is less than or equal to the depth update value of the k-th pixel, the computer device maintains the depth value of the target pixel unchanged. The position of the target pixel in the map to be updated matches the map position of the k-th pixel, and the k-th pixel in the (i+1)-th frame is any pixel in the (i+1)-th frame.

[0099] For example, assuming that the map position of the kth pixel (in the shadow space) in the i+1th frame is (x1, y1) and the depth value is z1, then the pixel at (x1, y1) in the map to be updated is the target pixel, and the depth value of the target pixel is set to z2; if z1 is greater than or equal to z2, the depth value z2 of the target pixel remains unchanged; if z1 is less than z2, the depth value of the target pixel is updated to z1.

[0100] By comparing the depth value of the target pixel in the map to be updated with the kth pixel that matches its position in the jth shadow map, the pixel belonging to the shadow part can be quickly determined in the map to be updated, thereby improving the efficiency of generating the i+1th frame cascade shadow map.

[0101] S310 , based on the first map and the second map corresponding to the j-th shadow map in the cascaded shadow maps of the i-th frame, update the j-th shadow map in the cascaded shadow maps of the i-th frame to obtain the (i+1)-th frame cascaded shadow map.

[0102] The j-th shadow map in the cascaded shadow map of the i+1th frame is obtained by updating the j-th shadow map in the cascaded shadow map of the i-th frame based on the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame. The i+1-th frame cascaded shadow map is used to render the i+1-th frame.

[0103] In one embodiment, a computer device fuses the first map and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame to obtain a fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; specifically, if any one of the first map and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, the computer device directly determines the depth value as the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; if both the first map and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame include the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, the computer device determines the smaller value therebetween as the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame.

[0104] For example, assuming that the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes: the depth value z1 of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame does not include: the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, then the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is determined to be z1. Similarly, assuming that the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame does not include: the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes: the depth value z2 of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, then the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is determined to be z2. Assume that the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes: the depth value z1 of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame includes: the depth value z2 of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; if z1 is greater than or equal to z2, then the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is determined to be z2, and if z1 is less than z2, then the depth value of the k-th pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame is determined to be z1.

[0105] Furthermore, the computer device performs filtering processing on the M fused maps to obtain M filtered fused maps. The filtering process is used to fill the depth value of the target pixel in the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame (the first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame do not include the depth value of the target pixel). Specifically, the computer device obtains the depth value of the adjacent pixel points of the target pixel from the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, and predicts the depth value of the target pixel based on the depth value of the adjacent pixel points of the target pixel. Among them, the adjacent pixel points of the target pixel may refer to the pixel points located within a preset step size (such as 2) around the target pixel point. Figure 4b is a schematic diagram of the adjacent pixel points of a target pixel provided in an embodiment of the present application. As shown in FIG4 b , when the preset step size is 2, the adjacent pixel points of the target pixel point A3B3 are: A1B1-A1B5, A2B1-A2B5, A3B1, A3B2, A3B4, A3B5, A4B1-A4B5 and A5B1-A5B5.

[0106] Optionally, the adjacent pixel points of the target pixel point may also refer to the pixel points that are closest to the target pixel point in the four directions of up, down, left, and right of the target pixel point and carry a depth value. Figure 4c is a schematic diagram of the adjacent pixel points of another target pixel point provided in an embodiment of the present application. As shown in Figure 4c, the adjacent pixel points of the target pixel point A3B3 are: A2B3, A3B1, A3B4 and A5B3. It will be understood that the depth values ​​of the various pixel points in Figures 4b and 4c are for example only and do not constitute an actual limitation of the present application.

[0107] Assume that the number of adjacent pixels of the target pixel is Q, where Q is an integer greater than 1. In one embodiment, the computer determines the median or minimum value among the depth values ​​of the Q adjacent pixels as the depth value of the target pixel. Taking Figure 4b as an example, the computer device sorts the depth values ​​of the pixels in Figure 4b in order from small to large or from large to small, and the median value of the depth values ​​of the pixels in Figure 4b is 16, then the computer device replaces the depth value of the target pixel with 16. Taking Figure 4c as an example, the depth values ​​of the adjacent pixels of the target pixel are 6, 7, 16, and 20 respectively; then the computer device sets the depth value of the target pixel to 6.

[0108] In one implementation, a computer device first groups the pixels in a fused map, with each group of pixels including P×P pixels, where P is an integer greater than 1. On the one hand, the computer device (e.g., through a compute shader) assigns a thread to each pixel in the group to read the depth value of the pixel (in the shadow space); the depth values ​​read by each thread are stored (e.g., stored in Group Shared Memory and the Group Memory Barrier With Group Sync function is called to synchronize the data stored in Group Shared Memory). On the other hand, the computer device calls one or more threads in each pixel group to read the depth values ​​of the adjacent pixels in the pixel group, and stores the depth values ​​read by each thread (e.g., stored in Group Shared Memory and the Group Memory Barrier With Group Sync function is called to synchronize the data stored in Group Shared Memory). The computer device then predicts the pixel using the depth values ​​of the adjacent pixels in the pixel group to obtain a filtered fused map.

[0109] For example, assuming that 16×16 pixels are grouped, the computer device allocates 256 threads to the group, and the preset step size is 2. For a pixel group, the scale of the pixel group and the adjacent pixels of the pixel group is 20×20. The computer device can call R (R is greater than or equal to 1 and less than or equal to 144) threads out of the allocated 256 threads to read the depth values ​​of the adjacent pixels of the pixel group, and predict the pixel through the depth values ​​of the adjacent pixels of one or more pixels in the pixel group to obtain the filtered fused map.

[0110] In another embodiment, the computer device determines the average value of the depth values ​​of Q adjacent pixels as the depth value of the target pixel. In yet another embodiment, the computer device predicts the depth values ​​of Q adjacent pixels through a prediction model to obtain the depth value of the target pixel. Optionally, the above-mentioned implementation method of filtering the target pixel can also be applied to other pixel points in the fused map (such as pixel points in the fused map that already carry depth values, that is, the depth values ​​of pixel points in the fused map that already carry depth values ​​are updated by the above-mentioned filtering method), and this application does not impose any restrictions on this.

[0111] Since the depth values ​​of adjacent pixels can reflect the depth value of the area from the overall level, the depth value of the target pixel can be predicted more accurately through the depth values ​​of adjacent pixels, thereby improving the integrity of the pixel depth values ​​in the fused map.

[0112] According to the implementation in steps S302 to S310 above, the computer device can obtain M filtered fused maps, and update the M shadow maps of the cascaded shadow map of the i-th frame through the M filtered fused maps (such as replacing the M shadow maps of the cascaded shadow map of the i-th frame with the corresponding M filtered fused maps) to obtain an updated cascaded shadow map of the i+1-th frame.

[0113] Practice has proved that the image processing method provided by this application (i.e., reconstructing the Shadow Map in the cascaded shadow map of the i+1th frame by reprojecting the Shadow Map in the cascaded shadow map of the i-th frame and the pixel points (positions) in the i-th picture frame) can greatly improve the quality and smoothness of the picture shadows when the light (light source in the scene) changes, compared with the traditional framed Shadow Map solution, while retaining the low time consumption and bandwidth characteristics of the traditional framed Shadow Map.

[0114] S311 , rendering each pixel in the (i+1)th picture frame by using the (i+1)th frame cascaded shadow map to obtain the (i+1)th picture frame.

[0115] In one embodiment, the computer device obtains a view projection matrix (of a capture device) corresponding to the i+1th frame and a shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map. Furthermore, the computer device determines the position of each pixel in the i+1th frame in view space based on the position of each pixel in the i+1th frame in the i+1th frame and the depth buffer information of the i+1th frame.

[0116] Furthermore, the computer device performs a third transformation on the position of each pixel in the i+1th frame in view space using the view projection matrix to obtain the position of each pixel in the i+1th frame in world space. Then, using the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map, the computer device performs a second transformation on the position of each pixel in the i+1th frame in world space to obtain the position of each pixel in the i+1th frame in shadow space. After obtaining the position of each pixel in the i+1th frame in shadow space, the computer device renders each pixel in the i+1th frame based on the position of each pixel in the i+1th frame in shadow space and the i+1th frame cascaded shadow map to obtain the i+1th frame.

[0117] In one embodiment, a computer device determines a mapping position and depth value of the kth pixel based on the position of the kth pixel in the shadow space in the i+1th frame. If the depth value of the kth pixel is greater than the depth value of the target pixel in the cascaded shadow map of the i+1th frame, the computer device determines that the kth pixel is a shadow area pixel; if the depth value of the kth pixel is less than or equal to the depth value of the target pixel in the cascaded shadow map of the i+1th frame, the computer device determines that the kth pixel is a non-shadow area pixel; wherein the position of the target pixel in the jth shadow map of the cascaded shadow map of the i+1th frame matches the mapping position of the kth pixel.

[0118] By comparing the depth value difference between the pixel points in the i+1th frame and the depth value difference between the matching pixels in the i+1th frame cascade shadow map, we can quickly determine which pixels in the i+1th frame belong to the shadow area and which belong to the non-shadow area, effectively improving the rendering efficiency of the i+1th frame and achieving better shadow rendering effects.

[0119] In one implementation, the image frame may be an image frame of a virtual scene; for example, the image frame may be a virtual flight image frame of an aircraft (such as a drone or airplane), or a virtual terrain detection image frame of a movable platform (such as an underwater robot or mountain exploration equipment). The computer device obtains a cascaded shadow map of the i-th frame of the virtual scene and performs a first reprojection process on the cascaded shadow map of the i-th frame to obtain M first maps. The computer device also obtains depth buffer information of the i+1-th frame and, based on the depth buffer information of the i+1-th frame, determines the position of each pixel in the i+1-th frame in view space. The computer device then performs a second reprojection process on the position of each pixel in the i+1-th frame of the virtual scene in view space to obtain M second maps corresponding to the M first maps. After obtaining M first maps and M second maps, the computer device updates the M shadow maps in the i-th frame cascaded shadow map based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map. The i+1-th frame of the virtual scene is then rendered using the i+1-th frame cascaded shadow map. This implementation method can simulate virtual scene frames (e.g., simulated aircraft flight frames) and improve the fidelity of the virtual scene frames.

[0120] In another implementation, the picture frame may be an animation frame in an animation; for example, the picture frame may be an animation frame in a three-dimensional game animation. The computer device obtains a cascaded shadow map of the i-th frame of the three-dimensional game animation and performs a first reprojection process on the cascaded shadow map of the i-th frame of the three-dimensional game animation to obtain M first maps. Furthermore, the computer device obtains depth buffer information of the i+1-th animation frame and, based on the depth buffer information of the i+1-th animation frame, determines the position of each pixel in the i+1-th animation frame in view space. The computer device then performs a second reprojection process on the position of each pixel in the i+1-th animation frame in view space to obtain M second maps corresponding to the M first maps. After obtaining the M first maps and the M second maps, the computer device updates the M shadow maps in the cascaded shadow map of the i-th frame based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map. The i+1-th frame animation frame of the three-dimensional game animation can be rendered using the i+1-th frame cascaded shadow map. It can be seen that this implementation method can be used to render three-dimensional game animations, and the rendering speed of game animation frames can be increased by reducing the computing resources required for rendering game animation frames, thereby improving the smoothness of three-dimensional games and improving the restoration (realism) of three-dimensional game animations.

[0121] In another implementation, the image frame can be a point cloud frame. The computer device obtains the cascaded shadow map of the i-th frame and performs a first reprojection process on the cascaded shadow map of the i-th frame to obtain M first maps. Furthermore, the computer device obtains the depth buffer information of the i+1-th point cloud frame and, based on the depth buffer information of the i+1-th point cloud frame, determines the position of each pixel in the i+1-th point cloud frame in the view space. The computer device then performs a second reprojection process on the position of each pixel in the i+1-th image frame in the view space to obtain M second maps corresponding to the M first maps. After obtaining the M first maps and the M second maps, the computer device updates the M shadow maps in the cascaded shadow map of the i-th frame based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map. The i+1-th frame point cloud frame can then be rendered using the i+1-th frame cascaded shadow map. It can be seen that the point cloud frame can be rendered through this implementation method, which effectively reduces the computing resources required for point cloud frame rendering and improves the restoration (realism) of the point cloud frame.

[0122] In an embodiment of the present application, a cascaded shadow map of the i-th frame is obtained, a first reprojection process is performed on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, depth buffer information of each pixel in the i+1-th frame is obtained, and a second reprojection process is performed on the pixels in the i+1-th frame based on the depth buffer information to obtain M second maps, and the M shadow maps are updated based on the M first maps and the M second maps to obtain a cascaded shadow map of the i+1-th frame. It can be seen that by updating the M shadow maps using the reprojection results of the M shadow maps in the cascaded shadow map of the i-th frame and the reprojection results of the depth buffer information of the i+1-th frame to obtain the cascaded shadow map of the i+1-th frame, the computing resources required for updating the cascaded shadow maps can be reduced; further, by rendering the i+1-th frame using the updated cascaded shadow map of the i+1-th frame, the rendering effect of the frame can be improved. In addition, in the process of updating the M shadow maps based on the M fused maps (obtained by fusing the M first maps and the M second maps), the smoothness of the fused map can be improved by filtering one or more pixels in the fused map, thereby improving the quality of the cascaded shadow map of the i+1th frame.

[0123] The above describes in detail the method of the embodiment of the present application. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0124] Please refer to Figure 5, which is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device shown in Figure 5 can be installed in a computer device, which can specifically be a terminal device or a server. The image processing device shown in Figure 5 can be used to perform some or all of the functions of the method embodiments described in Figures 2 and 3 above. Referring to Figure 5, the image processing device includes:

[0125] An acquiring unit 501 is configured to acquire an i-th frame cascaded shadow map, where the i-th frame cascaded shadow map is used to render an i-th picture frame, where i is a positive integer.

[0126] A processing unit 502 is configured to perform reprojection processing on M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer;

[0127] The acquisition unit 501 is further configured to acquire depth buffer information of each pixel in the (i+1)th picture frame;

[0128] The processing unit 502 is further configured to perform a second reprojection process on the pixels in the (i+1)th picture frame based on the depth buffer information to obtain M second maps;

[0129] The method is used to update the M shadow maps based on the M first maps and the M second maps to obtain the i+1-th frame cascaded shadow map; the i+1-th frame cascaded shadow map is used to render the i+1-th picture frame.

[0130] In one embodiment, each shadow map carries the depth value of each pixel in the shadow map; the processing unit 502 performs a first reprojection process on the M shadow maps in the cascaded shadow map of the i-th frame to obtain the M first maps, including:

[0131] Get the shadow projection matrix of the jth shadow map of the i+1th frame cascade shadow map;

[0132] Determine, based on the depth value of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame, the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space;

[0133] Update the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space through the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame;

[0134] Based on the updated positions of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space, the map to be updated is updated to obtain a first map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; the map to be updated is the j-th shadow map of the cascaded shadow map of the i-th frame, or a preset shadow map.

[0135] In one embodiment, the processing unit 502 is configured to update the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space using the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame, specifically to:

[0136] Get the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame;

[0137] Performing a first transformation on the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space by using the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame, thereby obtaining the position of each pixel in the j-th shadow map of the cascaded shadow map of the i-th frame in the world space;

[0138] Using the shadow projection matrix of the j-th shadow map of the i+1-th frame cascaded shadow map, a second transformation is performed on the position of each pixel in the j-th shadow map of the i-th frame cascaded shadow map in the world space to obtain the updated position of each pixel in the j-th shadow map of the i-th frame cascaded shadow map in the shadow space.

[0139] In one embodiment, the processing unit 502 updates the map to be updated based on the updated positions of each pixel in the j-th shadow map of the i-th frame cascaded shadow map in the shadow space, and obtains a first map corresponding to the j-th shadow map in the i-th frame cascaded shadow map, including:

[0140] Determine the updated map position and depth value of the k-th pixel in the j-th shadow map of the cascaded shadow map of the i-th frame based on the updated position of the k-th pixel in the shadow space;

[0141] If the depth value of the target pixel in the map to be updated is greater than the depth update value of the k-th pixel, the depth value of the target pixel is updated to the depth update value of the k-th pixel;

[0142] Among them, the position of the target pixel in the map to be updated matches the map position of the kth pixel; the kth pixel in the jth shadow map of the i-th frame cascade shadow map is any pixel in the jth shadow map of the i-th frame cascade shadow map.

[0143] In one embodiment, the processing unit 502 performs a second reprojection process on the pixels in the (i+1)th frame based on the depth buffer information to obtain M second maps, including:

[0144] Get the view projection matrix corresponding to the i+1th frame, and the shadow projection matrix of the jth shadow map of the i+1th frame cascade shadow map;

[0145] Determine the position of each pixel in the (i+1)th picture frame in the view space based on the position of each pixel in the (i+1)th picture frame in the (i+1)th picture frame and the depth buffer information;

[0146] Convert the position of each pixel in the i+1th frame in the view space to the position in the shadow space using the view projection matrix corresponding to the i+1th frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map;

[0147] Based on the position of each pixel point in the shadow space in the (i+1)th frame, the map to be updated is updated to obtain a second map corresponding to the jth shadow map in the cascaded shadow maps of the (i)th frame; the map to be updated is the jth shadow map in the cascaded shadow maps of the (i)th frame, or a preset shadow map.

[0148] In one embodiment, the processing unit 502 is configured to convert the position of each pixel in the (i+1)th picture frame in the view space into a position in the shadow space by using the view projection matrix corresponding to the (i+1)th picture frame and the shadow projection matrix of the j-th shadow map of the (i+1)th cascaded shadow map. Specifically, the processing unit 502 is configured to:

[0149] Performing a third transformation on the position of each pixel point in the (i+1)th picture frame in the view space using the view projection matrix to obtain the position of each pixel point in the (i+1)th picture frame in the world space;

[0150] The position of each pixel point in the i+1th frame in the world space is second transformed by the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map to obtain the position of each pixel point in the i+1th frame in the shadow space.

[0151] In one embodiment, the processing unit 502 updates the to-be-updated map based on the positions of each pixel in the shadow space in the (i+1)th frame to obtain a second map corresponding to the jth shadow map in the cascaded shadow maps of the (i)th frame, including:

[0152] Determine the texture position and depth value of the k-th pixel based on the position of the k-th pixel in the shadow space in the (i+1)-th frame;

[0153] If the depth value of the target pixel in the map to be updated is greater than the depth value of the k-th pixel, the depth value of the target pixel is updated to the depth value of the k-th pixel;

[0154] The position of the target pixel in the map to be updated matches the map position of the kth pixel, and the kth pixel in the (i+1)th frame is any pixel in the (i+1)th frame.

[0155] In one embodiment, the processing unit 502 is configured to update the M shadow maps based on the M first maps and the M second maps to obtain the (i+1)th frame cascaded shadow map, specifically to:

[0156] Fusing each first texture with the second texture corresponding to the first texture to obtain M fused textures;

[0157] Filtering the M fused maps to obtain M filtered fused maps;

[0158] The M shadow maps of the cascaded shadow map of the i-th frame are updated using the M filtered fused maps to obtain an updated cascaded shadow map of the i+1-th frame.

[0159] In one embodiment, the processing unit 502 performs filtering on the M fused maps to obtain the M filtered fused maps, including:

[0160] Obtain the depth value of the neighboring pixel of the target pixel from the fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, where the depth value of the target pixel is not included in the first map and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame;

[0161] The depth value of the target pixel is predicted based on the depth values ​​of the neighboring pixels of the target pixel.

[0162] In one embodiment, the number of neighboring pixels of the target pixel is Q, where Q is an integer greater than 1; the processing unit 502 is configured to predict the depth value of the target pixel based on the depth values ​​of the neighboring pixels of the target pixel, specifically to:

[0163] Determine the middle value or minimum value among the depth values ​​of Q adjacent pixels as the depth value of the target pixel; or

[0164] The average of the depth values ​​of Q adjacent pixels is determined as the depth value of the target pixel; or

[0165] The depth value of the target pixel is obtained by predicting the depth values ​​of Q adjacent pixels through the prediction model.

[0166] In one embodiment, the processing unit 502 is further configured to:

[0167] Get the view projection matrix corresponding to the i+1th frame, and the shadow projection matrix of the jth shadow map of the i+1th frame cascade shadow map;

[0168] Determine the position of each pixel in the (i+1)th picture frame in the view space based on the position of each pixel in the (i+1)th picture frame in the (i+1)th picture frame and the depth buffer information;

[0169] Performing a third transformation on the position of each pixel point in the (i+1)th picture frame in the view space using the view projection matrix to obtain the position of each pixel point in the (i+1)th picture frame in the world space;

[0170] Perform a second transformation on the position of each pixel in the i+1th frame in world space using the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map to obtain the position of each pixel in the i+1th frame in shadow space;

[0171] Based on the position of each pixel point in the i+1th picture frame in the shadow space and the i+1th frame cascade shadow map, each pixel point in the i+1th picture frame is rendered to obtain the i+1th picture frame.

[0172] In one embodiment, the processing unit 502 renders each pixel in the i+1th frame based on the position of each pixel in the i+1th frame in the shadow space and the i+1th frame cascaded shadow map to obtain the i+1th frame, including:

[0173] Determine the texture position and depth value of the k-th pixel based on the position of the k-th pixel in the shadow space in the (i+1)-th frame;

[0174] If the depth value of the k-th pixel is greater than the depth value of the target pixel in the i+1-th frame cascade shadow map, the k-th pixel is determined to be a shadow area pixel;

[0175] If the depth value of the k-th pixel is less than or equal to the depth value of the target pixel in the cascaded shadow map of the i+1-th frame, the k-th pixel is determined to be a non-shadow area pixel;

[0176] The position of the target pixel in the j-th shadow map of the cascaded shadow map of the i+1-th frame matches the map position of the k-th pixel.

[0177] According to one embodiment of the present application, some of the steps involved in the image processing method shown in Figures 2 and 3 can be performed by the various units in the image processing device shown in Figure 5. For example, step S201 shown in Figure 2 can be performed by the acquisition unit 501 shown in Figure 5, steps S202 and S204 can be performed by the processing unit 502 shown in Figure 5, and step S203 can be performed collaboratively by the acquisition unit 501 and the processing unit 502 shown in Figure 5; steps S301, S302, and S306 shown in Figure 3 can be performed by the acquisition unit 501 shown in Figure 5, and steps S303-S305 and steps S307-S311 can be performed by the processing unit 502 shown in Figure 5. The various units in the image processing device shown in Figure 5 can be individually or completely merged into one or more other units to form a composition, or one (or some) of the units can be further divided into multiple functionally smaller units to form a composition, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In actual applications, the functions of one unit can be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the image processing device can also include other units. In actual applications, these functions can also be assisted by other units and can be implemented by multiple units in collaboration.

[0178] According to another embodiment of the present application, an image processing device as shown in FIG5 can be constructed, and the image processing method of the embodiment of the present application can be implemented, by running a computer program (including program code) capable of executing the steps involved in the corresponding methods shown in FIG2 and FIG3 on a general-purpose computing device such as a computer device, which includes processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM). The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the computing device via the computer-readable recording medium and executed therein.

[0179] Based on the same inventive concept, the principles and beneficial effects of solving the problems by the image processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0180] Please refer to Figure 6, which is a structural diagram of a computer device provided in an embodiment of the present application. The computer device can be a terminal device or a server. As shown in Figure 6, the computer device includes at least a processor 601, a communication interface 602 and a memory 603. Among them, the processor 601, the communication interface 602 and the memory 603 can be connected via a bus or other means. Among them, the processor 601 (or the central processing unit (CPU)) is the computing core and control core of the computer device. It can parse various instructions within the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions issued by the object to the computer device and control the computer device to perform power on and off operations; for example, the CPU can transmit various interactive data between the internal structures of the computer device, etc. The communication interface 602 can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 601; the communication interface 602 can also be used for the transmission and interaction of data within the computer device. The memory 603 (Memory) is a memory device in the computer device for storing programs and data. It is understood that the memory 603 herein may include both the built-in memory of the computer device and, of course, the extended memory supported by the computer device. The memory 603 provides storage space for storing the operating system of the computer device, which may include, but is not limited to, Android, iOS, Windows Phone, etc., and this application does not limit this.

[0181] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the processing system of the computer device. In addition, a computer program suitable for being loaded and executed by the processor 601 is also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0182] In one embodiment, the processor 601 performs the following operations by running the computer program in the memory 603:

[0183] Get the cascaded shadow map of the i-th frame, the cascaded shadow map of the i-th frame is used to render the i-th frame, where i is a positive integer;

[0184] Perform a first reprojection process on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer;

[0185] Obtaining depth buffer information of each pixel in the (i+1)th frame, and performing a second reprojection process on the pixel in the (i+1)th frame based on the depth buffer information to obtain M second maps;

[0186] Based on the M first maps and the M second maps, the M shadow maps are updated to obtain an i+1-th frame cascaded shadow map; the i+1-th frame cascaded shadow map is used to render the i+1-th picture frame.

[0187] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the computer device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the image processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0188] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the image processing method of the above method embodiment.

[0189] An embodiment of the present application further provides a computer program product including a computer program, which, when executed on a computer, enables the computer to execute the image processing method provided in the above embodiment.

[0190] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0191] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0192] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0193] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of this application are still within the scope covered by the application.

Claims

1. An image processing method, the method being executed by a computer device, the method comprising: Obtaining an i-th frame cascaded shadow map, wherein the i-th frame cascaded shadow map is used to render an i-th picture frame, where i is a positive integer; Performing a first reprojection process on the M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer; Obtaining depth buffer information of each pixel in the (i+1)th picture frame, and performing a second reprojection process on the pixel in the (i+1)th picture frame based on the depth buffer information to obtain M second maps; Based on the M first maps and the M second maps, the M shadow maps are updated to obtain an i+1-th frame cascaded shadow map, and the i+1-th frame cascaded shadow map is used to render the i+1-th picture frame.

2. The method of claim 1, wherein each shadow map carries a depth value of each pixel in the shadow map; and the process of performing a first reprojection process on the M shadow maps in the i-th frame cascaded shadow map to obtain the M first maps comprises: Get the shadow projection matrix of the jth shadow map of the cascaded shadow map of the i+1th frame, 1≤j≤M; Determine, based on the depth value of each pixel in the j-th shadow map of the i-th frame cascaded shadow map, the position of each pixel in the j-th shadow map of the i-th frame cascaded shadow map in the shadow space; Update the position of each pixel point in the j-th shadow map of the cascaded shadow map of the i+1-th frame in the shadow space through the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame; Based on the updated positions of each pixel point in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow space, the map to be updated is updated to obtain a first map corresponding to the jth shadow map in the cascaded shadow map of the i-th frame, and the map to be updated is the jth shadow map of the cascaded shadow map of the i-th frame, or a preset shadow map.

3. The method of claim 2, wherein the updating of the position of each pixel in the j-th shadow map of the cascaded shadow map of the i+1-th frame in the shadow space by using the shadow projection matrix of the j-th shadow map of the cascaded shadow map of the i+1-th frame comprises: Obtaining the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame; Performing a first conversion on the position of each pixel point in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space through the shadow projection inverse matrix of the j-th shadow map of the cascaded shadow map of the i-th frame, so as to obtain the position of each pixel point in the j-th shadow map of the cascaded shadow map of the i-th frame in the world space; A second transformation is performed on the positions of each pixel in the jth shadow map of the cascaded shadow map of the i+1th frame in the world space through the shadow projection matrix of the jth shadow map of the cascaded shadow map of the i+1th frame to obtain the updated positions of each pixel in the jth shadow map of the cascaded shadow map of the i-th frame in the shadow space.

4. The method of claim 2, wherein the process of updating the map to be updated based on the updated positions of each pixel in the j-th shadow map of the i-th frame cascaded shadow map in the shadow space to obtain a first map corresponding to the j-th shadow map in the i-th frame cascaded shadow map comprises: Determine a map position and a depth update value of the k-th pixel point based on an updated position of the k-th pixel point in the j-th shadow map of the cascaded shadow map of the i-th frame in the shadow space; If the depth value of the target pixel in the map to be updated is greater than the depth update value of the kth pixel, then The depth value of the target pixel is updated to the depth update value of the kth pixel; Among them, the position of the target pixel point in the map to be updated matches the map position of the kth pixel point; the kth pixel point in the jth shadow map of the i-th frame cascaded shadow map is any pixel point in the jth shadow map of the i-th frame cascaded shadow map.

5. The method according to any one of claims 1 to 4, wherein the process of performing a second reprojection process on the pixel points in the (i+1)th picture frame based on the depth buffer information to obtain M second maps comprises: Get the view projection matrix corresponding to the i+1th frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map, 1≤j≤M; Determine the position of each pixel point in the i+1th picture frame in the view space based on the position of each pixel point in the i+1th picture frame in the i+1th picture frame and the depth buffer information; Convert the position of each pixel in the i+1th picture frame in the view space to the position in the shadow space by using the view projection matrix corresponding to the i+1th picture frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map; Based on the position of each pixel point in the shadow space in the i+1th picture frame, the map to be updated is updated to obtain a second map corresponding to the jth shadow map in the cascaded shadow map of the i-th frame, and the map to be updated is the jth shadow map of the cascaded shadow map of the i-th frame, or a preset shadow map.

6. The method of claim 5, wherein the position of each pixel point in the i+1th picture frame in the view space is converted into a position in the shadow space by using the view projection matrix corresponding to the i+1th picture frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map, comprising: Performing a third transformation on the position of each pixel point in the i+1th picture frame in the view space by using the view projection matrix to obtain the position of each pixel point in the i+1th picture frame in the world space; The position of each pixel point in the i+1th picture frame in the world space is secondly transformed through the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map to obtain the position of each pixel point in the i+1th picture frame in the shadow space.

7. The method of claim 5, wherein the process of updating the to-be-updated map based on the position of each pixel in the i+1th picture frame in the shadow space to obtain a second map corresponding to the jth shadow map in the i-th frame cascaded shadow map comprises: Determine a mapping position and a depth value of the k-th pixel point based on a position of the k-th pixel point in the shadow space in the i+1-th picture frame; If the depth value of the target pixel in the map to be updated is greater than the depth value of the k-th pixel, the depth value of the target pixel is updated to the depth value of the k-th pixel; The position of the target pixel in the map to be updated matches the map position of the k-th pixel, and the k-th pixel in the i+1-th frame is any pixel in the i+1-th frame.

8. The method according to any one of claims 1 to 7, wherein the updating of the M shadow maps based on the M first maps and the M second maps to obtain the (i+1)th frame cascaded shadow map comprises: Merge each first map with the second map corresponding to the first map to obtain M merged maps; Performing filtering on the M fused maps to obtain M filtered fused maps; The M shadow maps of the cascaded shadow map of the i-th frame are updated by using the M filtered fused maps to obtain an updated cascaded shadow map of the i+1-th frame.

9. The method according to claim 8, wherein the process of filtering the M fused maps to obtain M filtered fused maps comprises: Obtaining depth values ​​of neighboring pixels of a target pixel from a fused map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame, where the depth value of the target pixel is not included in the first map and the second map corresponding to the j-th shadow map in the cascaded shadow map of the i-th frame; The depth value of the target pixel is predicted based on the depth values ​​of the neighboring pixels of the target pixel.

10. The method according to claim 9, wherein the number of neighboring pixels of the target pixel is Q, and Q is an integer greater than 1; and predicting the depth value of the target pixel based on the depth values ​​of the neighboring pixels of the target pixel comprises: Determine the middle value or the minimum value among the depth values ​​of the Q adjacent pixels as the depth value of the target pixel; or, An average value of the depth values ​​of the Q adjacent pixels is determined as the depth value of the target pixel; or, The depth values ​​of the Q adjacent pixels are predicted using a prediction model to obtain the depth value of the target pixel.

11. The method according to any one of claims 1 to 10, further comprising: Obtaining the view projection matrix corresponding to the i+1th picture frame and the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map; Determine the position of each pixel point in the i+1th picture frame in the view space based on the position of each pixel point in the i+1th picture frame in the i+1th picture frame and the depth buffer information; Performing a third transformation on the position of each pixel point in the i+1th picture frame in the view space by using the view projection matrix to obtain the position of each pixel point in the i+1th picture frame in the world space; Performing a second transformation on the position of each pixel point in the i+1th picture frame in the world space through the shadow projection matrix of the jth shadow map of the i+1th frame cascaded shadow map, so as to obtain the position of each pixel point in the i+1th picture frame in the shadow space; Based on the position of each pixel point in the i+1th picture frame in the shadow space and the i+1th frame cascaded shadow map, each pixel point in the i+1th picture frame is rendered to obtain the i+1th picture frame.

12. The method according to claim 11, wherein the process of rendering each pixel in the i+1th picture frame based on the position of each pixel in the i+1th picture frame in the shadow space and the i+1th frame cascaded shadow map to obtain the i+1th picture frame comprises: Determine a mapping position and a depth value of the k-th pixel point based on a position of the k-th pixel point in the shadow space in the i+1-th picture frame; If the depth value of the k-th pixel is greater than the depth value of the target pixel in the (i+1)-th frame cascaded shadow map, the k-th pixel is determined to be a shadow area pixel; If the depth value of the k-th pixel is less than or equal to the depth value of the target pixel in the cascaded shadow map of the (i+1)-th frame, the k-th pixel is determined to be a non-shadow area pixel; The position of the target pixel in the j-th shadow map of the (i+1)-th frame cascaded shadow map matches the map position of the k-th pixel.

13. An image processing device, comprising: An acquisition unit, used for acquiring an i-th frame cascaded shadow map, wherein the i-th frame cascaded shadow map is used for rendering an i-th picture frame, where i is a positive integer; A processing unit, configured to perform reprojection processing on M shadow maps in the cascaded shadow map of the i-th frame to obtain M first maps, where M is a positive integer; The acquisition unit is further used to acquire the depth buffer information of each pixel in the (i+1)th picture frame; The processing unit is further configured to perform a second reprojection process on the pixel points in the (i+1)th picture frame based on the depth buffer information to obtain M second maps; And it is used to update the M shadow maps based on the M first maps and the M second maps to obtain the i+1th frame cascaded shadow map; the i+1th frame cascaded shadow map is used to render the i+1th picture frame.

14. A computer device comprising: Memory and processor; a memory, wherein a computer program is stored in the memory; A processor, used to load the computer program to implement the image processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the image processing method according to any one of claims 1 to 12.

Images