Method and apparatus for rendering a shadow image

Abstract

The application provides a shadow image rendering method and device, comprising: based on a virtual camera, collecting a plurality of to-be-rendered images including a target object; wherein the shadow of the target object included in the to-be-rendered image is continuously changed in the position of the target object in the image frame sequence formed by the plurality of to-be-rendered images; in the camera coordinate system, obtaining the position information and rotation information of the target object in each to-be-rendered image; based on the position information, determining the irradiation direction of the virtual light source in each to-be-rendered image, and based on the rotation information, determining the shadow offset in each to-be-rendered image; based on the irradiation direction, determining the first shadow of the target object in each to-be-rendered image, and based on the shadow offset and the first shadow, determining the second shadow; based on each second shadow, rendering the plurality of to-be-rendered images to obtain a plurality of shadow images. In this way, the rendering efficiency can be improved on the basis of ensuring the rendering effect of the shadow image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a shadow image rendering method and device. BACKGROUND

[0002] In the related art, common dynamic shadow calculation methods include Shadow Mapping based on a lighting system and Ray Traced Shadows. These technologies calculate shadows by simulating the light propagation and occlusion relationship between a light source and an object. However, these methods usually rely on a relatively complex lighting model and calculation, which not only limits the rendering efficiency of the shadow, but also requires the rendered shadow to always remain in a fixed direction relative to the assumed light source. However, in reality, if the object rotates, the real shadow will also rotate, which results in inconsistent performance between the rendered shadow and the real shadow, thereby leading to poor rendering effect. SUMMARY

[0003] Embodiments of the present application provide a shadow image rendering method, device, electronic equipment, computer readable storage medium and computer program product, which can improve rendering efficiency while ensuring the rendering effect of the shadow image.

[0004] The technical solutions of the embodiments of the present application are implemented as follows:

[0005] The embodiments of the present application provide a shadow image rendering method, which comprises the following steps:

[0006] Based on a virtual camera, a plurality of to-be-rendered images including a target object are collected;

[0007] The to-be-rendered images include shadows of the target object, and the position of the target object is continuously changed in an image frame sequence formed by the plurality of to-be-rendered images;

[0008] In a camera coordinate system, the position information and rotation information of the target object in each to-be-rendered image are obtained;

[0009] Based on the position information, the illumination direction of a virtual light source in each to-be-rendered image is determined, and based on the rotation information, a shadow offset is determined in each to-be-rendered image;

[0010] For each to-be-rendered image, based on the illumination direction, a first shadow of the target object in the to-be-rendered image is determined, and based on the shadow offset and the first shadow, a second shadow is determined;

[0011] Based on the second shadows, the plurality of to-be-rendered images are rendered to obtain a plurality of shadow images.

[0012] The embodiment of the present application provides a shadow image rendering device, and the device comprises:

[0013] The acquisition module is configured to acquire a plurality of to-be-rendered images including a target object based on a virtual camera; wherein the to-be-rendered images include shadows of the target object, and the positions of the target object are continuously changed in an image frame sequence formed by the plurality of to-be-rendered images.

[0014] The acquisition module is configured to acquire position information and rotation information of the target object in each to-be-rendered image in a camera coordinate system.

[0015] The first determination module is configured to determine an illumination direction of a virtual light source in each to-be-rendered image based on the position information, and determine a shadow offset in each to-be-rendered image based on the rotation information.

[0016] The second determination module is configured to determine a first shadow of the target object in each to-be-rendered image based on the illumination direction, and determine a second shadow based on the shadow offset and the first shadow.

[0017] The rendering module is configured to render the plurality of to-be-rendered images based on the second shadows to obtain a plurality of shadow images.

[0018] The embodiment of the present application provides an electronic device, which comprises:

[0019] The memory is configured to store executable instructions.

[0020] The processor is configured to execute the executable instructions stored in the memory to implement the shadow image rendering method provided by the embodiment of the present application.

[0021] The embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions. When the computer executable instructions are executed by a processor, the processor executes the shadow image rendering method provided by the embodiment of the present application.

[0022] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the shadow image rendering method provided by the embodiment of the present application.

[0023] The embodiment of the present application has the following beneficial effects:

[0024] After a plurality of to-be-rendered images including dynamic target objects are captured based on virtual cameras, position information and rotation information of the target objects in each to-be-rendered image are acquired, then the illumination direction of the virtual light source in each to-be-rendered image is determined based on the position information, and the shadow offset in each to-be-rendered image is determined based on the rotation information, so that the second shadow is determined based on the shadow offset and the first shadow, and the plurality of to-be-rendered images are rendered based on each second shadow to obtain a plurality of shadow images. In this way, compared with the scheme in the related art in which the rendered shadow and the real shadow do not perform consistently if the object rotates, the shadow offset in each to-be-rendered image is determined based on the rotation information, and the second shadow to be rendered is determined based on the shadow offset and the first shadow of the target object determined based on the illumination direction, so that the rendering effect of the shadow image is ensured, and meanwhile, the rendering process does not depend on a complex light model and calculation, and the rendering efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is an architecture schematic diagram of a shadow image rendering system 100 provided by an embodiment of the present application;

[0026] Figure 2 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0027] Figure 3 is a flow schematic diagram of a shadow image rendering method provided by an embodiment of the present application;

[0028] Figure 4 is a flow schematic diagram of determining a first shadow provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor shall fall within the scope of protection of the present application.

[0030] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0031] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0033] Before the embodiments of the present application are further described, the terms and names involved in the embodiments of the present application are explained, and the terms and names involved in the embodiments of the present application are applicable to the following explanations.

[0034] 1) Artificial intelligence (AI), is to use digital computers or digital computer controlled machine simulation, extension and expansion of human intelligence, perception of the environment, knowledge acquisition and use of knowledge to obtain the best results of theory, methods, techniques and application systems. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies.

[0035] 2) Client, also known as user end, is a program corresponding to the server to provide local service for users. Except for some applications that can only run locally, it is generally installed on a common client and needs to run with the server, that is, it needs a corresponding server and service program in the network to provide corresponding services. Therefore, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application program.

[0036] 3) Shader, is a program running on a graphics processing unit (GPU), which determines the appearance, color, lighting, shadow and other visual effects of objects in the scene. By directly interacting with the pixel processing pipeline of the GPU, the color and brightness of each pixel and the surface details of the object are controlled.

[0037] Reference Figure 1 , Figure 1 is an architecture diagram of a shadow image rendering system 100 provided by the embodiments of the present application. In order to realize the application scenario of rendering the shadow image, the terminal (exemplarily shows a terminal 400) connects the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. The terminal 400 is used for a user to use the client 401 to display on the display interface (exemplarily shows a display interface 401-1). The terminal 400 and the server 200 are connected to each other through wired or wireless network.

[0038] The server 200 is configured to collect a plurality of to-be-rendered images including a target object based on a virtual camera; a shadow of the target object included in the to-be-rendered images is continuously changed in position of the target object in a sequence of image frames formed by the plurality of to-be-rendered images; position information and rotation information of the target object in each to-be-rendered image are obtained in a camera coordinate system; an illumination direction of a virtual light source in each to-be-rendered image is determined based on the position information, and a shadow offset is determined in each to-be-rendered image based on the rotation information; for each to-be-rendered image, a first shadow of the target object in the to-be-rendered image is determined based on the illumination direction, and a second shadow is determined based on the shadow offset and the first shadow; the plurality of to-be-rendered images are rendered based on the second shadows to obtain a plurality of shadow images; and the plurality of shadow images are sent to the terminal 400.

[0039] The terminal 400 is configured to display the plurality of shadow images.

[0040] In some embodiments, the server 200 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, etc. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a set-top box, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and a mobile device (such as a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, a smart speaker, and a smart watch), etc., but is not limited thereto. The terminal device and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0041] Referring to Figure 2 , Figure 2 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. In actual application, the electronic device can be Figure 1 the server 200 or the terminal 400 shown, referring to Figure 2 , Figure 2 The electronic device includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the electronic device are coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, only the data bus is shown in Figure 2Various buses are labeled as a bus system 440.

[0042] The processor 410 can be an integrated circuit chip that has a processing capability of signals, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0043] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432 that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0044] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 optionally includes one or more storage devices physically located in proximity to the processor 410.

[0045] The memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. Non-volatile memory can be read only memory (ROM), and volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0046] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are exemplarily illustrated below.

[0047] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0048] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, and exemplary network interfaces 420 include Bluetooth, wireless compatibility authentication (WiFi), and universal serial bus (USB), etc.

[0049] a presentation module 453 configured to enable presentation of information (e.g., a user interface for operating a peripheral device and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430;

[0050] an input processing module 454 configured to detect and interpret one or more user inputs or interactions from one or more input devices 432.

[0051] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in software, Figure 2 A rendering device 455 of the shadow image stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a collection module 4551, an acquisition module 4552, a first determination module 4553, a second determination module 4554, and a rendering module 4555. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0052] In other embodiments, the apparatus provided by the embodiments of the present application can be implemented in hardware. As an example, the rendering device of the shadow image provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to perform the shadow image rendering method provided by the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can use one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic elements.

[0053] In some embodiments, a terminal or server can implement the shadow image rendering method provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or a software module in an operating system; can be a native (Native) application (APP), i.e., a program that needs to be installed in an operating system to run, such as an instant messaging APP, a web browser APP; can also be a mini-program, i.e., a program that only needs to be downloaded into a browser environment to run; and can also be a mini-program that can be embedded into any APP. In summary, the above computer program can be any form of application program, module or plug-in.

[0054] Based on the above description of the rendering system of the shadow image and the electronic device provided by the embodiments of the present application, the rendering method of the shadow image provided by the embodiments of the present application is described below. In actual implementation, the rendering method of the shadow image provided by the embodiments of the present application can be implemented by a terminal or a server alone, or by a terminal and a server cooperatively, so as to execute the rendering method of the shadow image provided by the embodiments of the present application by the terminal or the server alone or by the terminal and the server cooperatively. Figure 1 The rendering method of the shadow image provided by the embodiments of the present application is described below by taking the case that the rendering method of the shadow image provided by the embodiments of the present application is executed by the server 200 alone in the system 100. Figure 3 Figure 3 is a flowchart of the rendering method of the shadow image provided by the embodiments of the present application. Next, the steps shown in Figure 3 are described in combination with

[0055] In step 101, the server collects a plurality of to-be-rendered images including a target object based on a virtual camera; wherein the shadow of the target object included in the to-be-rendered image is continuously changed in position of the target object in an image frame sequence formed by the plurality of to-be-rendered images.

[0056] In actual implementation, the target object is dynamic, so that the position of the target object in the image frame sequence formed by the plurality of to-be-rendered images is continuously changed; wherein the target object being dynamic means that the target object moves in a virtual scene corresponding to the virtual camera, for example, the target object is an image element with a moving ability in the to-be-rendered image, that is, a virtual object with a moving ability in the virtual scene, such as an animal, a person, etc.

[0057] In actual application, the virtual scene includes a virtual light source, and the shadow of the target object included in the to-be-rendered image refers to a shadow area generated by the virtual light source illuminating the target object in the virtual scene.

[0058] It should be noted that the virtual light source is a graphical element for providing lighting effects in the virtual scene, which is obtained by simulating a real light source in the virtual scene; and the virtual camera is a component for simulating the function of a real camera, which determines the observation angle of the virtual scene by simulating the shooting function of the real camera, defines the position and direction of the virtual object controlled by the user in the virtual environment, that is, defines the scene content of the virtual scene observed by the virtual object controlled by the user; and the virtual scene can be a virtual game scene or a video and audio live scene, etc.

[0059] ​In actual implementation, before the server collects the plurality of to-be-rendered images including the target object based on the virtual camera, the server first acquires an image rendering request sent by the terminal, and the image rendering request is used to request to acquire a plurality of shadow images in the virtual scene. Specifically, taking a virtual game scene as the virtual scene, first, the terminal displays the virtual scene, and when the view angle of the player role controlled by the terminal, that is, the view angle of the virtual camera, includes the target object, the terminal sends an image rendering request to the server; thus, the server responds to the image rendering request and collects the plurality of to-be-rendered images including the target object based on the virtual camera.

[0060] In step 102, the position information and the rotation information of the target object in each to-be-rendered image are acquired in the camera coordinate system.

[0061] It should be noted that the camera coordinate system is a reference system used to describe the position and direction of the virtual camera in the three-dimensional space, which is defined from the perspective of the camera and is used to simulate the view angle and field of view when the camera observes the three-dimensional scene. The camera coordinate system is usually composed of the following key elements: origin, z-axis (vertical axis), x-axis (horizontal axis), and y-axis (vertical axis). The origin of the camera coordinate system is the center position of the camera itself, which is usually located at the focal point of the camera lens. The z-axis of the camera coordinate system points to the front of the camera, that is, the direction of the camera, which is usually aligned with the optical axis of the camera. In perspective projection, the z-axis determines the direction of the line of sight. The x-axis of the camera coordinate system usually points to the right side of the camera, which is perpendicular to the z-axis and forms a right-handed coordinate system (in some cases, such as OpenGL, a left-handed coordinate system may be used). The y-axis of the camera coordinate system usually points to the top of the camera, which is perpendicular to the x-axis and the z-axis, completing the definition of the right-handed coordinate system. The position information of the target object can be the coordinates of the target object in the camera coordinate system, and the rotation information is the angle of rotation of the target object in the camera coordinate system.

[0062] In actual implementation, the process of acquiring the position information of the target object in each to-be-rendered image can be to acquire the default color attribute of the target object; and to decrypt the color value on the target dimension of the default color attribute to obtain the position information of the target object.

[0063] It should be noted that for some parameters that need to be transmitted into the shader (such as the positions of the object and the light source, the rotation of the object, and the like), the default color attribute of the game object can be used to transmit the parameters, the color attribute can consist of four values (r, g, b, a), and the target dimension corresponding to the position refers to any two of the four dimensions, such as obtaining the default color attribute (0, 0, 0, 0), and after the target object moves, the position of the target object (156, 234) is converted into 0.156, 0.234, that is, the position of the target object is assigned to the r and g of the color attribute respectively, and then the obtained default color attribute is decrypted in the shader to obtain the position information of the target object.

[0064] In actual implementation, the process of obtaining the rotation information of the target object in each to-be-rendered image can be that the color attribute of the target object is obtained; the color value on the target dimension of the color attribute is decrypted to obtain the rotation information of the target object.

[0065] It should be noted that, as described above, for some parameters that need to be transmitted into the shader (such as the positions of the object and the light source, the rotation of the object, and the like), the default color attribute of the game object can be used to transmit the parameters, the color attribute can consist of four values (r, g, b, a), and the target dimension corresponding to the position refers to any two of the four dimensions, such as obtaining the default color attribute (0, 0, 0, 0), and after the target object moves, the position of the target object (156, 234) is converted into 0.156, 0.234, that is, the position of the target object is assigned to the r and g of the color attribute respectively, and then the obtained default color attribute is decrypted in the shader to obtain the position information of the target object.

[0066] In actual implementation, since there are multiple to-be-rendered images collected, a parallel processing manner can be used to render the shadow image, that is, the parallel processing manner is used to obtain the position information and the rotation information of the target object in each to-be-rendered image; specifically, the dynamic batch rule of Unity is used to splice the multiple to-be-rendered images that need to be rendered during game running to obtain a spliced atlas (which is equivalent to splicing the multiple to-be-rendered images into one large image, so that the large image is directly processed, that is, the to-be-rendered images in different regions of the large image are processed, so that the parallel processing process is realized), and then the atlas is placed under the same canvas, and the atlas is rendered with the shadow image, so that Unity automatically processes the to-be-rendered images in different regions of the atlas into one rendering batch and transmits it to the GPU to realize parallel processing.

[0067] Thus, first, by the parallel computing capability of the GPU, the shadow is generated and updated in real time in the process of the movement of a large number of image elements, that is, the target object, solving the problem that the traditional lighting system relies heavily on shadow simulation and cannot efficiently process large-scale image dynamic changes, that is, the change of large-scale dynamic shadow can be calculated in real time without the participation of the traditional lighting system, not only significantly reducing the consumption of computing resources and improving the rendering efficiency, but also effectively avoiding the calculation bottleneck when rendering a large number of image elements (such as multiple moving objects on the screen), ensuring the smoothness of real-time rendering. The shadow calculation of each object can be independent and simultaneous, without mutual interference, thereby improving the rendering throughput; second, by using the batch mechanism of the Unity engine, when rendering a large number of similar objects, for example, multiple similar images or objects, they can be placed under the same canvas level, that is, they can be combined into one rendering batch. This batch merging method reduces the state switching during rendering, further improving the rendering efficiency, especially in the case of limited hardware performance, which can significantly improve the performance.

[0068] In step 103, the illumination direction of the virtual light source in each to-be-rendered image is determined based on the position information, and the shadow offset in each to-be-rendered image is determined based on the rotation information.

[0069] In actual implementation, the process of determining the illumination direction of the virtual light source in each to-be-rendered image based on the position information can be, for each to-be-rendered image, performing the following processing: obtaining the position information of the virtual light source in the camera coordinate system; based on the position information of the virtual light source and the position information of the target object, the direction of the virtual light source pointing to the target object is taken as the illumination direction of the virtual light source in the to-be-rendered image.

[0070] It should be noted that first, the position information of the virtual light source and the position information of the target object are obtained in the world coordinate system, and the position information of the virtual light source and the position information of the target object are converted from the world coordinate system to the camera coordinate system, so that the position information of the virtual light source and the position information of the target object are obtained in the camera coordinate system.

[0071] The world coordinate system is used to define the positions and directions of all objects in a three-dimensional scene, and is a global reference frame that provides a common positioning reference for all objects in the three-dimensional scene. The world coordinate system is a right-handed coordinate system, which is usually composed of the following key elements: origin, z-axis (vertical axis), x-axis (horizontal axis) and y-axis (vertical axis), wherein the origin is the reference point in the world coordinate system, and its coordinates are (0, 0, 0). The positions of all objects are defined relative to this origin; the x-axis is a horizontal axis, usually pointing to the right or left (the specific direction depends on the definition, but usually to the right), which is used to represent the position of the object in the horizontal direction; the y-axis is also a horizontal axis, usually pointing forward or backward (the specific direction depends on the definition, but usually forward), which is used to represent the position of the object in the forward and backward directions; the z-axis is a vertical axis, usually pointing upwards. It is used to represent the position of the object in the vertical direction.

[0072] It should be noted that, as described above, the position information is equivalent to the coordinates, so the process of converting the position information of the virtual light source and the position information of the target object from the world coordinate system to the camera coordinate system can be to obtain a view matrix, which defines the position and orientation (i.e., the view angle) of the virtual camera. Then, based on the view matrix, the coordinates of the virtual light source and the coordinates of the target object in the world coordinate system are converted to the camera coordinate system to obtain the coordinates of the virtual light source and the coordinates of the target object in the camera coordinate system, i.e., the position information of the virtual light source and the position information of the target object in the camera coordinate system.

[0073] Then, based on the position information of the virtual light source and the position information of the target object, the direction of the virtual light source pointing to the target object is determined as the irradiation direction of the virtual light source in the to-be-rendered image, which can be based on the position information of the virtual light source and the position information of the target object to determine the coordinates of the virtual light source and the coordinates of the target object; subtracting the coordinates of the target object from the coordinates of the virtual light source to obtain a vector pointing from the virtual light source to the target object, and determining the direction indicated by the vector as the irradiation direction of the virtual light source in the to-be-rendered image.

[0074] In actual implementation, based on the rotation information, the process of determining the shadow offset in each to-be-rendered image can be to perform the following processing for each to-be-rendered image to obtain the shadow offset in the to-be-rendered image: based on the rotation information, determine the rotation angle of the target object in the to-be-rendered image; obtain the cosine value and the sine value corresponding to the rotation angle, and based on the cosine value and the sine value, determine the coordinate offset; multiply the coordinate offset by the pre-set shadow length to obtain the shadow offset, i.e.:

[0075] y = (cos(θ), sin(θ)) * shadowLength …… Equation (1);

[0076] wherein y is a shadow offset, (cos(θ), sin(θ)) is a coordinate offset, shadowLength is a shadow length, and θ is a rotation angle, i.e. an angle between a direction of the virtual light source and a horizontal plane (i.e. x-axis).

[0077] It should be noted that in the movement process of the target object, the position, rotation, and other information of the target object are transmitted to the shader every frame, so that after the cosine value corresponding to the rotation angle is obtained based on the rotation information, the cosine value is determined as the horizontal coordinate and the sine value is determined as the vertical coordinate, so that the coordinate offset (cos(θ), sin(θ)) is determined, and the pre-set shadow length is a pre-set constant. Herein, the embodiments of the present application are not limited.

[0078] In step 104, for each to-be-rendered image, a first shadow of the target object in the to-be-rendered image is determined based on the direction of illumination, and a second shadow is determined based on the shadow offset and the first shadow.

[0079] In actual implementation, for each to-be-rendered image, the process of determining the first shadow of the target object in the to-be-rendered image based on the direction of illumination is described with reference to Figure 4 , Figure 4 is a flowchart of determining the first shadow provided by the embodiments of the present application, and the process of determining the first shadow of the target object in the to-be-rendered image based on the direction of illumination can be realized by the following steps. Figure 4

[0080] It should be noted that in a three-dimensional scene, each scene element, i.e. object, corresponds to a plurality of vertices, and the vertex is the basis for constructing a three-dimensional object model, i.e. a basic unit constituting the surface of the object, which is used to represent the geometric shape and surface properties of the object; wherein each vertex has a three-dimensional coordinate (x, y, z), which determines the position of the vertex in space, and by connecting the vertices, a triangle or quadrilateral can be formed, and these polygons are combined to form the surface of the object; wherein the vertex data in the to-be-rendered image can be directly obtained by the shader.

[0081] It should be noted that in a three-dimensional scene, each scene element, i.e. object, corresponds to a plurality of vertices, and the vertex is the basis for constructing a three-dimensional object model, i.e. a basic unit constituting the surface of the object, which is used to represent the geometric shape and surface properties of the object; wherein each vertex has a three-dimensional coordinate (x, y, z), which determines the position of the vertex in space, and by connecting the vertices, a triangle or quadrilateral can be formed, and these polygons are combined to form the surface of the object; wherein the vertex data in the to-be-rendered image can be directly obtained by the shader.

[0082] In step 1042, based on the unit vector, a first vertex in the shadow region is selected from the plurality of vertices, and a first shadow is formed based on the first vertex.

[0083] ​In actual implementation, based on the unit vector, the process of selecting the first vertex in the shadow area from the plurality of vertices can be, in the camera coordinate system, determining the coordinates of each vertex; for each vertex, the following processing is performed to obtain the first vertex: performing dot product processing on the coordinates of the vertex and the unit vector to obtain a dot product result; when the dot product result is less than or equal to zero, the vertex is determined as the first vertex in the shadow area.

[0084] It should be noted that for each vertex of the target object, after performing dot product processing on the coordinates of the vertex and the unit vector to obtain a dot product result, if the dot product result is less than or equal to zero, it indicates that the vertex is not under the illumination of the light source, that is, the vertex is in the shadow area; if the dot product result is greater than zero, it indicates that the vertex is under the illumination of the light source. Thus, by traversing all vertices, the entire shadow area can be determined. Specifically, the dot product processing on the coordinates of the vertex and the unit vector to obtain the dot product result is:

[0085] z=P*D=x P *x D +y P *y D +z P *z D ……Equation (2);

[0086] wherein P is the vertex, the coordinates of which are (x P , y P , z P ), and the unit vector of the illumination direction of the virtual light source is D=(x D , y D , z D ).

[0087] In this way, by customizing the Shader to modify the underlying rendering logic, bypassing the traditional lighting system of game engines such as Unity, and directly utilizing the GPU for shadow calculation, the calculation complexity and rendering times can be reduced by optimizing the Shader code, thereby realizing more efficient dynamic shadow simulation.

[0088] In actual implementation, after determining the first vertex in the shadow area, based on the first vertex, a first shadow is formed, that is, generated, wherein the first shadow is composed of the first vertex.

[0089] In some embodiments, based on the shadow offset and the first shadow, the process of determining the second shadow can be, determining a plurality of first vertices included in the first shadow; for each first vertex, obtaining the coordinates of the first vertex and summing the coordinates of the first vertex with the shadow offset to obtain a second vertex; based on the second vertex corresponding to each first vertex, forming the second shadow.

[0090] It should be noted that the shadow offset is equivalent to a vector in which the shadow extends in the light direction, and the process of determining the second shadow based on the shadow offset and the first shadow, that is, extending or moving the first shadow by the shadow offset, to obtain the second shadow, wherein extending or moving the first shadow by the shadow offset means extending or moving all vertices of the shadow region by a distance corresponding to the shadow offset along the light direction, and adding the shadow offset to the position of the object, that is, summing the coordinates of the first vertex, to obtain the position of the shadow, that is:

[0091] S = P + y = P + (cos(θ), sin(θ)) * shadowLength …… Equation (3) ;

[0092] Wherein, S is the position of the shadow, P is the position of the target object, that is, the coordinates of the first vertex on the target object in the shadow region in actual calculation, y is the shadow offset in equation (1), (cos(θ), sin(θ)) is the coordinate offset, shadowLength is the shadow length, and θ is the rotation angle.

[0093] For example, assuming that the coordinates of the first vertex on the target object in the shadow region are (0, 0, 0), the light source is located at the right of the object, which means that θ is 0 (0 degree), so (cos(θ), sin(θ)) will be (1, 0), and assuming that shadowLength is 10, then the position of the shadow, that is, the second vertex corresponding to the first vertex S = (0, 0, 0) + (cos(θ), sin(θ)) * shadowLength = (0, 0, 0) + (1, 0) * 10 = (10, 0, 0), which means that the shadow will be at a position 10 units of distance to the left of the target object.

[0094] It should be noted that the first shadow is the shadow region of the target object that is not in the light region, and the second shadow is the shadow region corresponding to the target object formed when the virtual light source irradiates the target object, for example, when the target object is on the ground, the second shadow is the shadow region formed on the ground when the virtual light source irradiates the target object.

[0095] Step 105, based on each second shadow, rendering a plurality of to-be-rendered images to obtain a plurality of shadow images.

[0096] In actual implementation, based on each second shadow, the process of rendering the plurality of to-be-rendered images to obtain the plurality of shadow images can be, for each to-be-rendered image, acquiring a texture image corresponding to the to-be-rendered image; based on the texture image, determining a shadow intensity of an image pixel point in the to-be-rendered image in the second shadow and a color value of an image pixel point in the to-be-rendered image outside the second shadow; and based on the shadow intensity of the image pixel point in the to-be-rendered image in the second shadow and the color value of the image pixel point outside the second shadow, rendering to obtain a corresponding shadow image.

[0097] It should be noted that the process of acquiring the texture image corresponding to the to-be-rendered image can be, acquiring attribute information (including position, size, shape, etc.) of a virtual object in the to-be-rendered image in a model space corresponding to a virtual scene; based on the attribute information of the virtual object, determining at least two vertices of the virtual object in the model space, and constructing a space coordinate system of the corresponding model space, and in the space coordinate system, determining the coordinates and attributes (including normal, texture coordinates, tangent, etc.) of each vertex; performing coordinate conversion on the coordinates of each vertex in the space coordinate system to obtain the coordinates of each vertex under a screen coordinate system; and taking the coordinates of each vertex under the screen coordinate system and the attributes of each vertex as texture information of the first virtual object.

[0098] The screen coordinate system is a two-dimensional reference system for describing the position of a touch point (such as a cursor) on a physical device (such as a display); the origin of the screen coordinate system is located at the upper left corner of the screen, the X-axis is positive to the right, the Y-axis is positive downward, and the unit of measurement is pixel, which is used to locate a point on the screen corresponding to the terminal, such as the position of a mouse click, the position of a window, etc.

[0099] Then, based on the texture information of the virtual object, rasterization is performed to obtain a texture image corresponding to the to-be-rendered image, that is, an uncolored two-dimensional image of the to-be-rendered image in the screen space; wherein the rasterization here is used to convert the geometric data of the virtual object into pixel data, that is, to convert vector graphics (such as straight lines and curves) into raster images composed of pixels.

[0100] In actual implementation, based on the texture image, the process of determining the shadow intensity of the image pixel point in the to-be-rendered image in the second shadow and the color value of the image pixel point outside the second shadow can be, for each image pixel point in the texture image, based on the texture information of the virtual object, performing lighting calculation (that is, lighting rendering and shading) to obtain the shadow intensity of the image pixel point in the to-be-rendered image in the second shadow and the color value of the image pixel point outside the second shadow.

[0101] Then, based on the shadow intensity of the image pixel points in the second shadow in the to-be-rendered image and the color value of the image pixel points outside the second shadow, a corresponding shadow image is rendered; in this way, a plurality of shadow images are obtained.

[0102] In actual implementation, rendering the plurality of to-be-rendered images to obtain the plurality of shadow images can also optimize the shadow rendering process, so as to obtain the plurality of shadow images with better shadow rendering effect. Next, taking three optimization modes as examples, the process of optimizing the shadow rendering process is described.

[0103] In some embodiments, after determining the second shadow based on the shadow offset and the first shadow, the following processing can also be performed for each to-be-rendered image: performing convolution processing on the to-be-rendered image to obtain a horizontal gradient of the to-be-rendered image in the horizontal direction and a vertical gradient of the to-be-rendered image in the vertical direction; determining an edge region in the second shadow based on the horizontal gradient and the vertical gradient, and performing soft processing on the edge region in the second shadow to obtain a third shadow; and thus, the process of rendering the plurality of to-be-rendered images based on each second shadow to obtain the plurality of shadow images can be the process of rendering the plurality of to-be-rendered images based on each third shadow to obtain the plurality of shadow images.

[0104] It should be noted that since gradient calculation is usually performed on a grayscale image, the to-be-rendered image needs to be converted into a grayscale image first, so that the process of performing convolution processing on the to-be-rendered image to obtain a horizontal gradient of the to-be-rendered image in the horizontal direction and a vertical gradient of the to-be-rendered image in the vertical direction specifically includes: performing grayscale processing on the to-be-rendered image to obtain a grayscale image; and then performing convolution processing on the grayscale image to obtain a horizontal gradient of the grayscale image in the horizontal direction and a vertical gradient of the grayscale image in the vertical direction.

[0105] It should be noted that the edge of the picture can be detected in the shader, and then mapped to the edge of the shadow, that is, the area that needs to be processed in softness, and then the area is processed in softness, that is, blurred, so that the shadow edge is more soft and natural, and the unreal feeling caused by the hard shadow is reduced.

[0106] In actual implementation, first, two 3x3 convolution kernels are obtained, which are respectively used to detect the edges in the horizontal and vertical directions, and then the grayscale image is convolved based on the two convolution kernels to obtain the brightness gradients of the grayscale image in the horizontal and vertical directions. Specifically, the grayscale image is convolved based on the horizontal convolution kernel and the vertical convolution kernel to obtain the horizontal gradient and the vertical gradient, that is:

[0107] I x = I * G x ……Equation (4);

[0108] I y I*G y ……equation (5);

[0109] wherein I x is a horizontal gradient, G x is a convolution kernel for detecting edges in the horizontal direction, I y is a vertical gradient, G y is a convolution kernel for detecting edges in the vertical direction, and I is a gray-scale image corresponding to the image to be rendered.

[0110] It should be noted that, for the convolution processing of the gray-scale image based on the horizontal convolution kernel and the vertical convolution kernel, the horizontal gradient and the vertical gradient obtained are the horizontal gradient and the vertical gradient of each pixel point in the gray-scale image.

[0111] In actual implementation, based on the horizontal gradient and the vertical gradient, the process of determining the edge region in the second shadow can be that, for each pixel point in the gray-scale image, the square of the horizontal gradient of the pixel point is summed with the square of the vertical gradient of the pixel point to obtain a summation result; the summation result is square-rooted to obtain the edge intensity of the pixel point, that is:

[0112]

[0113] wherein E is the edge intensity of the pixel point, I x is a horizontal gradient, I y is a vertical gradient.

[0114] Then, target pixel points with edge intensity greater than an edge intensity threshold are determined from the plurality of pixel points, and a region where the target pixel points are located is taken as the edge region in the image to be rendered; wherein the edge intensity threshold is pre-set, and the embodiments of the present application do not limit this.

[0115] In actual implementation, after the edge region in the image to be rendered is determined, the process of performing soft processing on the edge region in the second shadow to obtain the third shadow can be that, the edge region is subjected to blur processing, such as Gaussian blur processing, to obtain the blurred second shadow, that is, the third shadow.

[0116] Then, based on the third shadows, the plurality of images to be rendered are rendered to obtain a plurality of shadow images. Here, the process of rendering the plurality of images to be rendered based on the third shadows to obtain the plurality of shadow images is similar to the process of rendering the plurality of images to be rendered based on the second shadows to obtain the plurality of shadow images described above, and the embodiments of the present application do not repeat the description.

[0117] Thus, after detecting the edge region of the image to be rendered, by performing the blur processing on the edge region, the shadow region at the edge can be made more natural and soft, and the unreal feeling caused by the hard shadow is reduced.

[0118] In some embodiments, after determining the second shadow based on the shadow offset and the first shadow, for each image to be rendered, the following processing can be performed: obtaining the distance between the target object and the virtual light source and the mapping relationship, the mapping relationship being used to indicate the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object; determining the target shadow adjustment coefficient corresponding to the target object based on the mapping relationship and the distance between the target object and the virtual light source; determining the shadow intensity of the second shadow, and adjusting the shadow intensity of the second shadow based on the target shadow adjustment coefficient to obtain a fourth shadow; thus, the process of rendering the plurality of images to be rendered based on the second shadow to obtain the plurality of shadow images can be the process of rendering the plurality of images to be rendered based on the fourth shadow to obtain the plurality of shadow images.

[0119] It should be noted that the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object, i.e., the mapping relationship, is pre-set. The shadow adjustment coefficient herein is used to adjust the shadow intensity of the shadow region corresponding to the corresponding object, so as to adjust the shadow intensity of the shadow region of different objects based on the shadow adjustment coefficient. If the object is relatively close to the virtual light source, the shadow intensity of the shadow region of the object is enhanced based on the corresponding shadow adjustment coefficient. If the object is relatively far from the virtual light source, the shadow intensity of the shadow region of the object is weakened based on the corresponding shadow adjustment coefficient.

[0120] It should be noted that the process of rendering the plurality of images to be rendered based on the fourth shadow to obtain the plurality of shadow images is similar to the process of rendering the plurality of images to be rendered based on the second shadow to obtain the plurality of shadow images as described above. For this, the embodiments of the present application do not repeat the description.

[0121] Thus, according to the distance between different objects and the light source, different levels of shadow intensity are simulated, the shadow near the object is darker, the shadow far from the object is gradually faded, and a more realistic shadow effect is created.

[0122] In some embodiments, the number of target objects is one, and the process of determining the second shadow based on the shadow offset and the first shadow includes: determining the second shadow based on the shadow offset and the first shadow.

[0123] It should be noted that if the to-be-rendered image includes multiple target objects, the two target objects corresponding to the overlapping shadow region are determined as a target object pair. The number of target object pairs can be one or more, and embodiments of the present application do not limit the number of target object pairs. Specifically, the multiple target objects are combined two by two to obtain multiple object combinations. From the multiple object combinations, an object combination including two target objects corresponding to the overlapping region of the second shadow is selected, that is, an object combination including two target objects corresponding to the overlapping shadow region is selected as a target object pair.

[0124] Then, for each target object pair, the shadow intensity of the overlapping shadow region of the two target objects included in the target object pair is obtained. Then, by comparing the two shadow intensities, a larger shadow intensity is selected from the two shadow intensities, and the larger shadow intensity is used as the shadow intensity of the overlapping shadow region of the corresponding two target objects. Based on the larger shadow intensity, the shadow intensity of the overlapping shadow region of each target object in the target object pair is updated to obtain the fifth shadow of each target object in the target object pair.

[0125] It should be noted that the process of rendering the multiple to-be-rendered images based on the fifth shadow of each target object in the target object pair and the second shadow of the other target objects to obtain the multiple shadow images is similar to the process of rendering the multiple to-be-rendered images based on the second shadow to obtain the multiple shadow images, and embodiments of the present application do not repeat the description.

[0126] Thus, when there are a large number of objects in the scene moving at the same time, there can be many objects that are very close to each other, and their shadows overlap and look very harsh, and cause a certain amount of overdraw, causing performance loss. Thus, by the feature that the shadow of an object farther from the assumed light source is lighter, if two objects have overlapping shadow areas, only the darker shadow is displayed, avoiding the problem of harsh display caused by shadow overlap, and reducing performance consumption.

[0127] It should be noted that, in the actual optimization of the shadow rendering process, at least one of the above optimization methods can be selected to optimize the shadow rendering process, and the embodiments of the present application are not limited thereto.

[0128] In some embodiments, before optimizing the shadow rendering process of the target object, the size of the target object can also be detected. If the detection result indicates that the size of the target object is greater than the target size, at least one of the above optimization methods is selected to optimize the shadow rendering process of the corresponding target object. If the size of the target object is less than or equal to the target size, because these small target objects do not affect the user's experience, the corresponding target object can not be optimized, wherein the target size is pre-set, and the embodiments of the present application are not limited thereto. Thus, the rendering efficiency and game performance can be further improved.

[0129] In some embodiments, the above embodiments of the present application can also be implemented through similar API interface calls, that is, the above embodiments can be ported to other engines that support custom Shaders, providing consistent rendering effects. Thus, the above embodiments of the present application can be universal on multiple rendering platforms, ensuring cross-platform compatibility.

[0130] By applying the above embodiments of the present application, after a plurality of to-be-rendered images including dynamic target objects are captured based on a virtual camera, position information and rotation information of the target objects in each to-be-rendered image are obtained, then based on the position information, the illumination direction of the virtual light source in each to-be-rendered image is determined, and based on the rotation information, the shadow offset in each to-be-rendered image is determined, so as to determine the second shadow based on the shadow offset and the first shadow, and then render the plurality of to-be-rendered images based on each second shadow to obtain a plurality of shadow images. Thus, compared with the scheme in the related art in which the rendered shadow and the real shadow do not perform consistently if the object rotates, the present application determines the shadow offset in each to-be-rendered image through the rotation information, and then determines the second shadow to be rendered based on the shadow offset and the first shadow of the target object determined based on the illumination direction, thereby ensuring the rendering effect of the shadow image; at the same time, the rendering process does not depend on a complex lighting model and calculation, and the rendering efficiency is also improved.

[0131] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0132] Among related technologies, common dynamic shadow calculation methods include shadow mapping based on lighting systems and ray-traced shadows. These techniques calculate shadows by simulating the propagation and occlusion relationships of light between light sources and objects. However, these methods typically rely on complex lighting models and calculations, especially in scenes with many moving objects, resulting in very high rendering costs. Therefore, these techniques are generally used for shadow rendering in large-scale 3D scene projects. In addition, the lighting systems provided by game engines such as Unity can also perform shadow calculations, but these systems typically rely on the Central Processing Unit (CPU) and GPU for a large amount of lighting and shadow calculations, which can easily lead to performance bottlenecks when rendering a large number of moving images on the same screen. Existing technologies often cannot maintain good image quality and smoothness on lower hardware configurations, especially on low-end mobile phones or devices.

[0133] Based on this, the shadow rendering methods for moving objects in related technologies have the following problems:

[0134] First, performance bottlenecks: Shadow calculation methods in related technologies (such as shadow mapping and ray tracing) lead to a sharp drop in performance when rendering scenes with highly dynamic and large-scale objects, especially on low-end devices, making it difficult to meet the needs of real-time rendering. The complexity and performance consumption of shadow calculations are particularly high when there are many moving objects on the screen.

[0135] Second, high computational resource consumption: Shadow calculation methods in related technologies usually rely on high computational resources, especially ray tracing technology, which requires a lot of computing time and GPU resources. It cannot meet the real-time requirements of low-end hardware devices, resulting in inconsistent performance of games or applications on different performance devices, affecting the user experience.

[0136] Third, reliance on lighting systems: Most related technologies rely on traditional lighting systems to simulate shadows, which limits the efficiency of real-time shadow updates and processing. Especially in the dynamic shadow rendering of large-scale moving objects, the computation of the lighting system often becomes a performance bottleneck.

[0137] Fourth, excessive rendering batches or overdraw: The shadow component provided by the Unity engine is implemented by rendering twice in a staggered manner, but this method will result in a very high proportion of overlapping areas in the rendering, which will become a performance bottleneck when large-scale object movement occurs.

[0138] Fifth, in some existing simulation scheme not dependent on illumination (such as the Shadow component provided by the development engine), the problem that the shadow needs to be kept in a fixed direction relative to the assumed light source during the rotation of the object is not solved. After the object rotates, the shadow will rotate together, resulting in inconsistent performance with the real shadow.

[0139] Therefore, the application provides a shadow rendering method, which aims to simulate the effect of dynamic shadow in the process of large-scale 2D image motion on the same screen through self-defined Shader and parallel computing, and does not depend on the illumination system of Unity, avoiding the dependence on the traditional illumination system. At the same time, an efficient dynamic shadow simulation algorithm is adopted to solve the problems of poor dynamic shadow calculation performance, high resource consumption and high hardware requirements in the prior art, so that the rendering process of dynamic shadow can be completed under lower calculation resource consumption, and the image quality in real-time rendering is guaranteed, so that users using middle and low-end mobile phones can also have a good experience. The specific solution is as follows:

[0140] 1. Through depth optimization of Shader, the efficient calculation capability of GPU is utilized to dynamically calculate the change of shadow, instead of relying on the traditional illumination model, thereby reducing the calculation burden in the rendering process, solving the problem of inconsistent shadow performance after image rotation, maintaining high rendering quality, efficiently processing shadow calculation of a large number of moving images, avoiding the performance bottleneck caused by a large number of calculations in the traditional technology, thereby significantly reducing the consumption of calculation resources and improving the rendering efficiency;

[0141] 2. Through the parallel computing capability of GPU, a rendering batch can be maintained when rendering different states of a large number of similar objects, thereby improving the rendering performance. Especially in a high-load scene, the shadow calculation of a large number of moving images can be processed in parallel, thereby greatly improving the throughput of the system;

[0142] 3. Not only can it be efficiently run in the Unity engine, but also can be applied to other game engines or rendering systems, having strong universality and expansibility, so that users can still obtain good dynamic shadow effect and smooth game experience even on devices with weak hardware performance.

[0143] In actual implementation, the application utilizes the parallel computing capability of GPU to optimize the shadow rendering process through self-defined Shader, breaks through the limitation of the traditional illumination system, and solves the problems of low dynamic shadow rendering efficiency and large resource consumption in the related art. The following is the detailed technical implementation process:

[0144] First, simulate the shadow by using the geometric data of the object

[0145] According to the contour of the picture (the shader can obtain the vertex data of the picture) and the assumed light source direction, it is determined which regions are in the shadow area of the picture through vector operation and dot product. Specifically, vector operation and dot product can be used to calculate whether each vertex of the object is under the illumination of the light source. Let the vertex of the object be P = (x P , y P , z P ), and the light source direction be D = (x D , y D , z D ), wherein D is a unit vector, and the dot product formula is shown in the above formula (2), so if P*D≤0, the vertex is in the shadow area. By traversing all the vertices, the entire shadow area can be determined.

[0146] Second, shadow offset based on object motion

[0147] During the motion of the object, its position, rotation, and the like are transmitted to the shader every frame, and the position of the shadow can be obtained according to the following formula, that is:

[0148] S = y + (cos(θ), sin(θ))*shadowLength …… formula (7);

[0149] wherein y is the shadow area on the object determined in the first step, (cos(θ), sin(θ)) is the coordinate offset, shadowLength is the pre-set shadow length, and θ is the rotation angle, that is, the angle between the illumination direction of the virtual light source and the horizontal plane (that is, the x-axis).

[0150] Third, softness adjustment of the shadow

[0151] In actual implementation, the edge of the picture can be detected in the shader, and then can be mapped to the edge of the shadow, that is, the area that needs to be processed in softness, and then the area is processed in softness, that is, blurred, so that the shadow edge is softer and more natural, and the unreal feeling caused by the hard shadow is reduced.

[0152] wherein the edge is detected by calculating the spatial gradient of the image brightness. Two 3x3 convolution kernels are first obtained for detecting the horizontal and vertical edges, and then the corresponding gray image of the image to be rendered is convolved based on the two convolution kernels, so that the brightness gradient of the gray image in the horizontal and vertical directions can be obtained. Specifically, the corresponding gray image of the image to be rendered is convolved based on the horizontal and vertical convolution kernels, so that the horizontal and vertical gradients can be obtained, as shown in the above formula (4) and formula (5).

[0153] Then, the edge intensity of each pixel point is determined based on the above formula (6), so that the target pixel point with the edge intensity greater than the edge intensity threshold is determined from the plurality of pixel points, and the region where the target pixel point is located is taken as the edge region in the image to be rendered; wherein the edge intensity threshold is pre-set, and the embodiments of the present application are not limited thereto. Then, after the edge region is determined, softness processing, i.e. blur processing, such as Gaussian blur processing, is performed on the edge region.

[0154] Fourth, multi-level shadow simulation

[0155] According to the distance of different objects from the light source, different levels of shadow intensity are simulated, the shadow near the place is darker, the shadow far away gradually fades, and a more realistic shadow effect is created.

[0156] Fifth, merged and superimposed shadows

[0157] When there are a large number of objects moving simultaneously in the scene, there can be many objects that are very close to each other, and their shadows are superimposed together, which looks very harsh and causes a certain Overdraw, resulting in performance loss. Therefore, by taking advantage of the feature that the shadow of an object farther away from the light source is lighter, if two objects have overlapping shadow regions, only the darker shadow is displayed, avoiding the problem of harsh display effect caused by shadow superposition, and reducing performance consumption.

[0158] Sixth, optimization of rendering performance

[0159] By using the dynamic batching rule of Unity, a plurality of images to be rendered are spliced to obtain a spliced atlas (equivalent to splicing a plurality of images to be rendered into one large image, so that the large image is processed directly, i.e. the images to be rendered in different regions of the large image are processed, thereby realizing parallel processing), and then the atlas is placed under the same canvas, and the atlas is rendered with a shadow image, so that Unity automatically processes the images to be rendered in different regions of the atlas into one rendering batch and delivers it to the GPU, thereby realizing parallel processing.

[0160] In addition, for some parameters (such as the positions of objects and light sources, the rotation of objects, etc.) that need to be transmitted to the shader, the Property module provided by the modified shader is not used, because this will cause Unity batching to fail, but the color attribute of the game object by default can be used to transmit parameters. For example, the color consists of four values (r, g, b, a), the parameters to be transmitted are simplified and encrypted, such as converting the position information (156, 234) to 0.156, 0.234, and assigning them to the r and g of the color attribute respectively, and then decrypting the obtained color attribute in the shader to obtain the real information.

[0161] Seventh, LOD (Level of Detail) optimization

[0162] In actual implementation, the size of the object (target object) can also be detected. If the detection result indicates that the size of the target object is greater than the target size, the processes of steps two, three, and four are selected to process the corresponding target object. If the size of the target object is less than or equal to the target size, because these small target objects do not affect the user's experience, the processes of steps two, three, and four can not be selected to process the corresponding target object. In this way, the rendering efficiency and game performance can be further improved.

[0163] Seventh, cross-platform adaptation

[0164] The above embodiments can also be implemented through similar API interface calls, that is, the above embodiments can be transplanted into other engines that support custom Shaders, providing consistent rendering effects. In this way, the above embodiments of the present application can be universal on multiple rendering platforms, ensuring cross-platform compatibility.

[0165] In this way, through the above embodiments, the custom Shader can effectively simulate dynamic shadow effects without relying on real-time lighting, while maintaining high performance and good user experience. These methods combine the geometry data, motion information, and screen space technology of the object to create more realistic and optimized shadow effects. Specifically, the custom Shader and GPU parallel computing model of the present application achieve deep optimization of the shadow calculation process, effectively reducing the consumption of computing resources, especially reducing the dependence on CPU, so that efficient real-time rendering can be maintained even in high-load image scenarios. At the same time, the rendering batch merging strategy improves rendering efficiency while reducing system resource requirements, ensuring performance optimization. Moreover, the adaptive rendering strategy introduced can automatically adjust the shadow quality according to device performance, so that the game can run smoothly on devices with lower performance, ensuring the consistency of user experience. In addition, the present application is independent of the core algorithm and Shader optimization design of Unity and other engines, supports cross-platform rendering applications, and can be applied to other mainstream graphics engines, also has wide versatility and expansibility.

[0166] In actual implementation, there are also some expansion or alternative ways based on the present application to increase its flexibility and applicability. For example:

[0167] 1. More flexible shadow generation: Real-time rendering of dynamic shadows can be completed through parallel computing. An "approximate shadow" generation method based on the preset motion trajectory of an object can be considered, especially in scenarios with low requirements for shadow accuracy. This method can further reduce the GPU occupancy by predicting the shadow based on the overall trend of the object's motion.

[0168] 2. Expansion of applicable data types: Although the main description of this application is the dynamic shadow rendering of moving objects in images, the same technical process and rendering idea can be applied to other types of multimedia data processing, such as dynamic light simulation in video frames, to improve the versatility of rendering.

[0169] 3. Cross-platform interface optimization: This application can be further refined to adapt to more rendering engine interfaces, such as developing a version specifically adapted to Unreal or other 3D engines. By optimizing the cross-platform rendering interface, the compatibility between engines can be improved, and more extensive applications on different devices can be provided.

[0170] 4. Compatibility with 2D and 3D rendering: By further expanding the parallel computing model, the dynamic shadow of a 3D object can be simulated and combined with a 2D image, facilitating rendering in 3D / 2D mixed scenes and enhancing the compatibility of the solution.

[0171] After the above embodiments of the present application are applied, based on the virtual camera capturing a plurality of to-be-rendered images including a dynamic target object, the position information and rotation information of the target object in each to-be-rendered image are obtained. Then, based on the position information, the illumination direction of the virtual light source in each to-be-rendered image is determined, and based on the rotation information, the shadow offset in each to-be-rendered image is determined. Thus, based on the shadow offset and the first shadow, the second shadow is determined, and then based on each second shadow, the plurality of to-be-rendered images are rendered to obtain a plurality of shadow images. In this way, compared with the solution in the related art in which the rendered shadow and the real shadow do not match if the object rotates, the present application determines the shadow offset in each to-be-rendered image through the rotation information, and then determines the second shadow to be rendered based on the shadow offset and the first shadow of the target object determined based on the illumination direction, thereby ensuring the rendering effect of the shadow image. At the same time, the rendering process does not depend on a complex lighting model and calculation, and the rendering efficiency is also improved.

[0172] The following continues to illustrate an exemplary structure of the implementation of the shadow image rendering device 455 provided by the embodiments of the present application as a software module. In some embodiments, as shown in FIG. 4B, the software module in the shadow image rendering device 455 stored in the memory 450 can include: Figure 2

[0173] ​The collection module 4551 is configured to collect a plurality of to-be-rendered images including a target object based on a virtual camera, wherein the to-be-rendered images include shadows of the target object, and in an image frame sequence formed by the plurality of to-be-rendered images, positions of the target object are continuously changed.

[0174] The acquisition module 4552 is configured to acquire position information and rotation information of the target object in each of the to-be-rendered images in a camera coordinate system.

[0175] The first determination module 4553 is configured to determine an illumination direction of a virtual light source in each of the to-be-rendered images based on the position information, and determine a shadow offset in each of the to-be-rendered images based on the rotation information.

[0176] The second determination module 4554 is configured to, for each of the to-be-rendered images, determine a first shadow of the target object in the to-be-rendered image based on the illumination direction, and determine a second shadow based on the shadow offset and the first shadow.

[0177] The rendering module 4555 is configured to render the plurality of to-be-rendered images based on the second shadows to obtain a plurality of shadow images.

[0178] In some embodiments, the first determination module 4553 is further configured to, for each of the to-be-rendered images, perform the following processing: acquire position information of the virtual light source in the camera coordinate system; and determine a direction in which the virtual light source points to the target object as the illumination direction of the virtual light source in the to-be-rendered image based on the position information of the virtual light source and the position information of the target object.

[0179] In some embodiments, the first determination module 4553 is further configured to, for each of the to-be-rendered images, perform the following processing to obtain the shadow offset in the to-be-rendered image: determine a rotation angle of the target object in the to-be-rendered image based on the rotation information; acquire a cosine value and a sine value corresponding to the rotation angle, and determine a coordinate offset based on the cosine value and the sine value; and obtain the shadow offset by multiplying the coordinate offset and a preset shadow length.

[0180] In some embodiments, the second determination module 4554 is further configured to acquire a unit vector in the illumination direction, and determine a plurality of vertices corresponding to the target object; select a first vertex in a shadow area from the plurality of vertices based on the unit vector, and form the first shadow based on the first vertex.

[0181] In some embodiments, the second determining module 4554 is further configured to determine coordinates of each of the vertices in the camera coordinate system; and for each of the vertices, perform the following processing to obtain the first vertex: perform dot product processing on the coordinates of the vertex and the unit vector to obtain a dot product result; and when the dot product result is less than or equal to zero, determine the vertex as the first vertex in the shadow area.

[0182] In some embodiments, the second determining module 4554 is further configured to determine a plurality of first vertices included in the first shadow; for each of the first vertices, obtain coordinates of the first vertex, and perform summation processing on the coordinates of the first vertex and the shadow offset to obtain a second vertex; and form the second shadow based on the second vertex corresponding to each of the first vertices.

[0183] In some embodiments, the apparatus further includes a first shadow adjusting module, which is configured to, for each of the to-be-rendered images, perform the following processing: perform convolution processing on the to-be-rendered image to obtain a horizontal gradient of the to-be-rendered image in a horizontal direction and a vertical gradient of the to-be-rendered image in a vertical direction; determine an edge region in the second shadow based on the horizontal gradient and the vertical gradient, and perform soft processing on the edge region in the second shadow to obtain a third shadow; and the rendering module 4555 is further configured to render the plurality of to-be-rendered images based on the third shadow to obtain a plurality of shadow images.

[0184] In some embodiments, the apparatus further includes a second shadow adjusting module, which is configured to, for each of the to-be-rendered images, perform the following processing: obtain a distance between the target object and the virtual light source and a mapping relationship, the mapping relationship being used to indicate a corresponding relationship between a distance between an object and a virtual light source and a shadow adjusting coefficient corresponding to the object; determine a target shadow adjusting coefficient corresponding to the target object based on the mapping relationship and the distance between the target object and the virtual light source; determine a shadow intensity of the second shadow, and adjust the shadow intensity of the second shadow based on the target shadow adjusting coefficient to obtain a fourth shadow; and the rendering module 4555 is further configured to render the plurality of to-be-rendered images based on the fourth shadow to obtain a plurality of shadow images.

[0185] In some embodiments, the apparatus further includes a third shadow adjustment module configured to, for each of the images to be rendered, perform the following processing: selecting a target object pair from the plurality of target objects, wherein the target object pair includes two target objects having a coincident shadow region; determining a shadow intensity of the coincident shadow region of each target object in the target object pair, and selecting a greater shadow intensity from the two shadow intensities; updating the shadow intensity of the coincident shadow region of each target object in the target object pair based on the greater shadow intensity, to obtain a fifth shadow of each target object in the target object pair; and the rendering module 4555 is further configured to render the plurality of images to be rendered based on the fifth shadow of each target object in the target object pair and the second shadow of other target objects, to obtain a plurality of shadow images, wherein the other target objects are target objects in the plurality of target objects other than the target objects in the target object pair.

[0186] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the electronic device performs the shadow image rendering method or the position matching method provided in the embodiment of the present application, for example, as shown in the shadow image rendering method. Figure 3 The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the electronic device performs the shadow image rendering method or the position matching method provided in the embodiment of the present application, for example, as shown in the shadow image rendering method.

[0187] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, cause the processor to perform the shadow image rendering method or the position matching method provided in the embodiment of the present application, for example, as shown in the shadow image rendering method. Figure 3 The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions, when executed by a processor, cause the processor to perform the shadow image rendering method or the position matching method provided in the embodiment of the present application, for example, as shown in the shadow image rendering method.

[0188] In some embodiments, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a CD-ROM, etc. The computer readable storage medium can also be various devices including one or any combination of the above storage devices.

[0189] In some embodiments, the executable instructions can take the form of programs, software, software modules, scripts, or code, written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and they can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0190] By way of example, the executable instructions can or can not correspond to a file in a file system, can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.

[0191] By way of example, the executable instructions can be deployed to be executed on one electronic device or on multiple electronic devices that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0192] It should be noted that, in the embodiments of the present application, related data such as video data including target objects and face scanning data are acquired, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0193] The above merely illustrates the embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A method of rendering a shadow image, characterized by, The method comprises: acquiring a plurality of to-be-rendered images including a target object based on a virtual camera; wherein the to-be-rendered images include shadows of the target object, and in a sequence of image frames formed by the plurality of to-be-rendered images, positions of the target object are continuously changed; obtaining position information and rotation information of the target object in each of the to-be-rendered images in a camera coordinate system; determining an irradiation direction of a virtual light source in each of the to-be-rendered images based on the position information; for each of the to-be-rendered images, performing the following processing to obtain a shadow offset in the to-be-rendered image: determining a rotation angle of the target object in the to-be-rendered image based on the rotation information; obtaining a cosine value and a sine value corresponding to the rotation angle, and taking the cosine value as an abscissa and the sine value as an ordinate to obtain a coordinate offset; and multiplying the coordinate offset and a pre-set shadow length to obtain the shadow offset, the shadow length being a pre-set constant; for each of the to-be-rendered images, determining a first shadow of the target object in the to-be-rendered image based on the irradiation direction, and determining a second shadow based on the shadow offset and the first shadow; rendering the plurality of to-be-rendered images based on the second shadows to obtain a plurality of shadow images.

2. The method of claim 1, wherein, The method comprises: for each of the to-be-rendered images, performing the following processing: obtaining position information of the virtual light source in the camera coordinate system; based on the position information of the virtual light source and the position information of the target object, taking a direction in which the virtual light source points to the target object as the irradiation direction of the virtual light source in the to-be-rendered image.

3. The method of claim 1, wherein, The method comprises: obtaining a unit vector in the irradiation direction and determining a plurality of vertices corresponding to the target object; based on the unit vector, selecting a first vertex in a shadow region from the plurality of vertices, and forming the first shadow based on the first vertex.

4. The method of claim 3, wherein, The method comprises: determining coordinates of each of the vertices in the camera coordinate system; for each of the vertices, performing the following processing to obtain the first vertex: performing dot product processing on the coordinates of the vertex and the unit vector to obtain a dot product result; when the dot product result is less than or equal to zero, determining the vertex as the first vertex in the shadow region.

5. The method of claim 1, wherein, The method comprises: determining a plurality of first vertices included in the first shadow; for each of the first vertices, obtaining coordinates of the first vertex and performing summation on the coordinates of the first vertex and the shadow offset to obtain a second vertex; forming the second shadow based on the second vertices corresponding to each of the first vertices.

6. The method of claim 1, wherein, After the second shadow is determined based on the shadow offset and the first shadow, the method further includes: For each of the to-be-rendered images, the following processing is performed: Convolution processing is performed on the to-be-rendered image to obtain a horizontal gradient of the to-be-rendered image in a horizontal direction and a vertical gradient in a vertical direction; Based on the horizontal gradient and the vertical gradient, an edge region in the second shadow is determined, and soft processing is performed on the edge region in the second shadow to obtain a third shadow; The rendering of the plurality of to-be-rendered images based on each of the second shadows to obtain a plurality of shadow images includes: The rendering of the plurality of to-be-rendered images based on each of the third shadows to obtain a plurality of shadow images.

7. The method of claim 1, wherein, After the second shadow is determined based on the shadow offset and the first shadow, the method further includes: For each of the to-be-rendered images, the following processing is performed: The distance between the target object and the virtual light source and the mapping relationship are obtained, and the mapping relationship is used to indicate the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object; Based on the mapping relationship and the distance between the target object and the virtual light source, a target shadow adjustment coefficient corresponding to the target object is determined; The shadow intensity of the second shadow is determined, and the shadow intensity of the second shadow is adjusted based on the target shadow adjustment coefficient to obtain a fourth shadow; The rendering of the plurality of to-be-rendered images based on each of the second shadows to obtain a plurality of shadow images includes: The rendering of the plurality of to-be-rendered images based on each of the fourth shadows to obtain a plurality of shadow images.

8. The method of claim 1, wherein, The number of target objects is a plurality; after the second shadow is determined based on the shadow offset and the first shadow, the method further includes: For each of the to-be-rendered images, the following processing is performed: A target object pair is selected from a plurality of target objects; wherein the target object pair includes two target objects that have a coincident shadow region; For each of the target object pairs, the shadow intensity of the coincident shadow region of each target object in the target object pair is determined, and a greater shadow intensity is selected from the two shadow intensities; based on the greater shadow intensity, the shadow intensity of the coincident shadow region of each target object in the target object pair is updated to obtain a fifth shadow of each target object in the target object pair; The rendering of the plurality of to-be-rendered images based on each of the second shadows to obtain a plurality of shadow images includes: The rendering of the plurality of to-be-rendered images based on the fifth shadow of each target object in the target object pair and the second shadow of other target objects to obtain a plurality of shadow images; The other target objects are target objects in the plurality of target objects, except for the target objects in the target object pair.

9. An apparatus for rendering a shadow image, characterized by The device includes: The collection module is configured to collect a plurality of to-be-rendered images including a target object based on a virtual camera, wherein the to-be-rendered images include shadows of the target object, and in an image frame sequence formed by the plurality of to-be-rendered images, positions of the target object are continuously changed. The acquisition module is configured to acquire position information and rotation information of the target object in each of the to-be-rendered images in a camera coordinate system. The first determination module is configured to determine an illumination direction of a virtual light source in each of the to-be-rendered images based on the position information, and for each of the to-be-rendered images, perform the following processing to obtain a shadow offset in the to-be-rendered image: determine a rotation angle of the target object in the to-be-rendered image based on the rotation information, acquire a cosine value and a sine value corresponding to the rotation angle, and obtain a coordinate offset by taking the cosine value as an abscissa and the sine value as an ordinate, multiply the coordinate offset by a pre-set shadow length to obtain the shadow offset, and the shadow length is a pre-set constant. The second determination module is configured to determine a first shadow of the target object in each of the to-be-rendered images based on the illumination direction, and determine a second shadow based on the shadow offset and the first shadow. The rendering module is configured to render the plurality of to-be-rendered images based on the second shadows to obtain a plurality of shadow images.

Images