Shadow image rendering method and device

Abstract

The invention provides a shadow image rendering method and device, and the method comprises the steps: collecting a plurality of to-be-rendered images comprising a target object based on a virtual camera; wherein the to-be-rendered images comprise the shadow of the target object, and the position of the target object in an image frame sequence formed by the plurality of to-be-rendered images is continuously changed; under a camera coordinate system, obtaining position information and rotation information of a target object in each to-be-rendered image; determining an irradiation direction of a virtual light source in each to-be-rendered image based on the position information, and determining a shadow offset in each to-be-rendered image based on the rotation information; determining a first shadow of a target object in each to-be-rendered image based on the irradiation direction, and determining a second shadow based on the shadow offset and the first shadow; and rendering the plurality of to-be-rendered images based on the second shadows to obtain a plurality of shadow images. Therefore, the rendering efficiency can be improved on the basis of ensuring the rendering effect of the shadow image.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and in particular, to a method and apparatus for rendering a shadow image. Background Art

[0002] In related technologies, common dynamic shadow calculation methods include Shadow Mapping and Ray Traced Shadows based on a lighting system. These technologies calculate shadows by simulating the light propagation and occlusion relationship between a light source and an object. However, these methods usually rely on relatively complex lighting models and calculations, which not only limit the rendering efficiency of shadows, but also the rendered shadows need to always remain in a fixed direction relative to a supposed light source. In practice, however, if an object rotates, the real shadow will rotate along with it. In this way, the rendered shadow and the real shadow behave inconsistently, resulting in a poor rendering effect. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for rendering a shadow image, which can improve the rendering efficiency on the basis of ensuring the rendering effect of the shadow image.

[0004] The technical solution of the embodiments of this application is implemented as follows:

[0005] Embodiments of this application provide a method for rendering a shadow image, the method including:

[0006] Collecting a plurality of images to be rendered including a target object based on a virtual camera;

[0007] Wherein, the images to be rendered include the shadow of the target object, and in the image frame sequence formed by the plurality of images to be rendered, the position of the target object changes continuously;

[0008] Obtaining position information and rotation information of the target object in each of the images to be rendered in a camera coordinate system;

[0009] Determining an illumination direction of a virtual light source in each of the images to be rendered based on the position information, and determining a shadow offset in each of the images to be rendered based on the rotation information;

[0010] For each of the images to be rendered, determining a first shadow of the target object in the image to be rendered based on the illumination direction, and determining a second shadow based on the shadow offset and the first shadow;

[0011] Rendering the plurality of images to be rendered based on each of the second shadows to obtain a plurality of shadow images.

[0012] An embodiment of the present application provides a rendering device for a shadow image, and the device includes:

[0013] An acquisition module, configured to acquire a plurality of to-be-rendered images including a target object based on a virtual camera; wherein, the to-be-rendered images include shadows of the target object, and in an image frame sequence formed by the plurality of to-be-rendered images, the position of the target object changes continuously;

[0014] An acquisition module, configured to acquire position information and rotation information of the target object in each of the to-be-rendered images in a camera coordinate system;

[0015] A first determination module, configured to determine an irradiation direction of a virtual light source in each of the to-be-rendered images based on the position information, and determine a shadow offset in each of the to-be-rendered images based on the rotation information;

[0016] A second determination module, configured to, for each of the to-be-rendered images, determine a first shadow of the target object in the to-be-rendered image based on the irradiation direction, and determine a second shadow based on the shadow offset and the first shadow;

[0017] A rendering module, configured to render the plurality of to-be-rendered images based on each of the second shadows to obtain a plurality of shadow images.

[0018] An embodiment of the present application provides an electronic device, including:

[0019] A memory, configured to store executable instructions;

[0020] A processor, configured to implement the shadow image rendering method provided by the embodiment of the present application when executing the executable instructions stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the processor will be caused to execute the shadow image rendering method provided by the embodiment of the present application.

[0022] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions, and the computer program or computer-executable instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium, and the processor executes the computer-executable instructions, so that the electronic device executes the shadow image rendering method provided by the embodiment of the present application.

[0023] The embodiment of the present application has the following beneficial effects:

[0024] After acquiring a plurality of to-be-rendered images including dynamic target objects based on a virtual camera, obtain the position information and rotation information of the target objects in each to-be-rendered image, then determine the illumination direction of the virtual light source in each to-be-rendered image based on the position information, and determine the shadow offset in each to-be-rendered image based on the rotation information, so as to determine the second shadow based on the shadow offset and the first shadow, and then render the plurality of to-be-rendered images based on each second shadow to obtain a plurality of shadow images. In this way, compared with the related art where the rendered shadow and the real shadow are inconsistent if the object rotates, in this application, the shadow offset is determined in each to-be-rendered image based on the rotation information, and then the second shadow to be rendered is determined based on the shadow offset and the first shadow of the target object determined based on the illumination direction, ensuring the rendering effect of the shadow image; at the same time, the rendering process does not rely on complex lighting models and calculations, and also improves the rendering efficiency. Description of the Drawings

[0025] Figure 1 is a schematic structural diagram of a shadow image rendering system 100 provided by an embodiment of the present application;

[0026] Figure 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0027] Figure 3 is a schematic flowchart of a shadow image rendering method provided by an embodiment of the present application;

[0028] Figure 4 is a schematic flowchart of determining the first shadow provided by an embodiment of the present application. Detailed Embodiments

[0029] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be construed as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0030] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0031] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are for the purpose of describing embodiments of this application only and are not intended to limit this application.

[0033] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are described. The nouns and terms involved in the embodiments of this application are subject to the following explanations.

[0034] 1) Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. Artificial intelligence technology is a comprehensive discipline with a wide range of fields, including both hardware-level and software-level technologies. Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics.

[0035] 2) A client, also known as a user terminal, refers to a program that provides local services corresponding to a server. Except for some applications that can only run locally, it is generally installed on ordinary client machines and needs to cooperate with the server to run, that is, corresponding servers and service programs in the network are required to provide corresponding services. Therefore, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application program.

[0036] 3) A Shader is a program that runs on a Graphics Processing Unit (GPU). It determines the appearance, color, lighting, shadows, and other visual effects of objects in a scene. By directly interacting with the pixel processing pipeline of the GPU, it controls the color and brightness of each pixel and the surface details of objects.

[0037] See Figure 1 , Figure 1 is a schematic diagram of the architecture of the shadow image rendering system 100 provided by the embodiments of this application. For the application scenario of realizing the rendering of shadow images, the terminal (exemplarily showing the terminal 400) is connected to the server 200 through the network 300. The network 300 can be a wide area network, a local area network, or a combination of both. The terminal 400 is used for the user to use the client 401 and display it on the display interface (exemplarily showing the display interface 401-1). The terminal 400 and the server 200 are connected to each other through wired or wireless networks.

[0038] Among them, the server 200 is used to collect a plurality of images to be rendered including the target object based on a virtual camera; among them, the images to be rendered include the shadow of the target object, and in the image frame sequence formed by the plurality of images to be rendered, the position of the target object changes continuously; in the camera coordinate system, obtain the position information and rotation information of the target object in each image to be rendered; based on the position information, determine the illumination direction of the virtual light source in each image to be rendered, and based on the rotation information, determine the shadow offset in each image to be rendered; for each image to be rendered, based on the illumination direction, determine the first shadow of the target object in the image to be rendered, and based on the shadow offset and the first shadow, determine the second shadow; based on each second shadow, render the plurality of images to be rendered to obtain a plurality of shadow images; send the plurality of shadow pictures to the terminal 400;

[0039] The terminal 400 is used to display a plurality of shadow pictures.

[0040] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, Content Deliver Network), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a set-top box, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, a smart speaker, and a smart watch), etc., but is not limited thereto. The terminal device and the server may be directly or indirectly connected through wired or wireless communication methods, and are not limited in the embodiments of the present application.

[0041] See Figure 2 , Figure 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. In actual applications, the electronic device may be Figure 1 the server 200 or the terminal 400 shown in Figure 2 , Figure 2 The electronic device shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the electronic device is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear description, in Figure 2Various buses are labeled as bus system 440.

[0042] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0043] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual display screens. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0044] Memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. Memory 450 optionally includes one or more storage devices that are physically located away from processor 410.

[0045] Memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0046] In some embodiments, memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0047] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0048] Network communication module 452, for reaching other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.;

[0049] A presentation module 453 for enabling the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.);

[0050] An input processing module 454 for detecting and translating one or more user inputs or interactions from one of one or more input devices 432.

[0051] In some embodiments, the device provided by the embodiments of the present application may be implemented in software. Figure 2 A rendering device 455 for a shadow image stored in the memory 450 is shown, which may be software in the form of a program, a plug-in, etc., including the following software modules: an acquisition module 4551, a obtaining module 4552, a first determination module 4553, a second determination module 4554, and a rendering module 4555. These modules are logical, and thus can be arbitrarily combined or further split according to the functions to be implemented. The functions of each module will be described hereinafter.

[0052] In other embodiments, the device provided by the embodiments of the present application may be implemented in hardware. As an example, the rendering device for a shadow image provided by the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the rendering method for a shadow image provided by the embodiments of the present application. For example, a processor in the form of a hardware decoding processor may employ one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0053] In some embodiments, a terminal or a server may implement the rendering method for a shadow image provided by the embodiments of the present application by running a computer program. For example, the computer program may be a native program or a software module in an operating system; it may be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP or a web browser APP; it may also be a small program, that is, a program that only needs to be downloaded into a browser environment to run; it may also be a small program that can be embedded into any APP. In short, the above computer program may be any form of application program, module, or plug-in.

[0054] Based on the above description of the shadow image rendering system and electronic device provided in the embodiments of the present application, the following describes the shadow image rendering method provided in the embodiments of the present application. In actual implementation, the shadow image rendering method provided in the embodiments of the present application can be implemented independently by a terminal or a server, or jointly implemented by a terminal and a server. Taking the server 200 in Figure 1 as an example to execute the shadow image rendering method provided in the embodiments of the present application alone for illustration. Refer to Figure 3 , Figure 3 which is a schematic flowchart of the shadow image rendering method provided in the embodiments of the present application. Below, the steps shown will be described in conjunction with Figure 3 .

[0055] Step 101, the server collects a plurality of images to be rendered including a target object based on a virtual camera; wherein, the images to be rendered include the shadow of the target object, and the position of the target object is continuously changing in the image frame sequence formed by the plurality of images to be rendered.

[0056] In actual implementation, the target object is dynamic, so that the position of the target object is continuously changing in the image frame sequence formed by the plurality of images to be rendered; wherein, the target object being dynamic means that the target object moves in the virtual scene corresponding to the virtual camera. For example, the target object is an image element with the ability to move in the image to be rendered, that is, a virtual object with the ability to move in the virtual scene, such as an animal, a person, etc.;

[0057] In practical applications, the virtual scene includes a virtual light source, and the shadow of the target object included in the image to be rendered refers to the shadow area generated by the virtual light source irradiating the target object in the virtual scene;

[0058] It should be noted that the virtual light source is a graphic element that provides a lighting effect in the virtual scene and is obtained by simulating a real light source in the virtual scene; while the virtual camera is a component that simulates the functions of a real camera, and determines the viewing angle of the virtual scene by simulating the shooting function of a real camera, defining the position and direction of the virtual object controlled by the user in the virtual environment, that is, used to determine the scene content of the virtual scene observed by the virtual object controlled by the user; and the virtual scene can be a virtual game scene, or an audio-video live broadcast scene, etc.

[0059] In actual implementation, before the server acquires multiple to-be-rendered images including the target object based on the virtual camera, the server first obtains an image rendering request sent by the terminal. The image rendering request is used to request to obtain multiple shadow images in the virtual scene. Specifically, taking the virtual scene as a virtual game scene as an example, first, the terminal displays the virtual scene. When the perspective of the player character controlled by the terminal, that is, the perspective of the virtual camera, includes the target object, the terminal sends an image rendering request to the server. Thus, in response to the image rendering request, the server acquires multiple to-be-rendered images including the target object based on the virtual camera.

[0060] Step 102, in the camera coordinate system, obtain the position information and rotation information of the target object in each to-be-rendered image.

[0061] It should be noted that the camera coordinate system is a reference system used to describe the position and orientation of the virtual camera in three-dimensional space. It is defined from the perspective of the camera and is used to simulate the perspective and field of view when the camera observes a three-dimensional scene. Among them, the camera coordinate system usually consists of the following key elements: the origin, the z-axis (vertical axis), the x-axis (horizontal axis), and the y-axis (longitudinal axis). The origin of the camera coordinate system is the central position of the camera itself, usually located at the focal point of the camera lens; the z-axis of the camera coordinate system points forward the camera, that is, the orientation of the camera, usually aligned with the optical axis of the camera. In perspective projection, the z-axis determines the line-of-sight direction; the x-axis of the camera coordinate system usually points to the right side of the camera, perpendicular to the z-axis, forming a right-handed coordinate system (in some cases, such as in OpenGL, a left-handed coordinate system may be used); the y-axis of the camera coordinate system usually points above the camera, perpendicular to the x-axis and the z-axis, completing the definition of the right-handed coordinate system; and the position information of the target object can be the coordinates of the target object in the camera coordinate system, while the rotation information is the rotation angle of the target object in the camera coordinate system.

[0062] In actual implementation, the process of obtaining the position information of the target object in each to-be-rendered image can be to obtain the default color attribute of the target object; decrypt the color value on the target dimension of the default color attribute to obtain the position information of the target object.

[0063] It should be noted that for some parameters that need to be passed into the shader (such as the positions of objects and light sources, the rotation of objects, etc.), the default color attribute of the game object can be used to pass the parameters. The color attribute can consist of 4 values (r, g, b, a), and the target dimension corresponding to the position refers to any two of these four dimensions. For example, the default color attribute obtained is (0, 0, 0, 0), and after the target object moves, the position of the target object such as (156, 234) is converted to 0.156, 0.234, that is, the position of the target object is respectively assigned to r and g of the color attribute, and then the default color attribute obtained is decrypted in the shader to obtain the position information of the target object.

[0064] In actual implementation, the process of obtaining the rotation information of the target object in each to-be-rendered image can be to obtain the color attribute of the target object; decrypt the color value on the target dimension of the color attribute to obtain the rotation information of the target object.

[0065] It should be noted that as described above, for some parameters that need to be passed into the shader (such as the positions of objects and light sources, the rotation of objects, etc.), the default color attribute of the game object can be used to pass the parameters. The color attribute can consist of 4 values (r, g, b, a), and the target dimension corresponding to the rotation angle refers to any one of these four dimensions. For example, the default color attribute obtained is (0, 0, 0, 0), and after the target object rotates, the rotation angle of the target object such as 30 degrees is converted to 0.30, that is, the rotation angle of the target object is respectively assigned to b of the color attribute, and then the default color attribute obtained is decrypted in the shader to obtain the rotation information of the target object.

[0066] In actual implementation, since there are multiple to-be-rendered images collected, therefore, a parallel processing method can be adopted to render the shadow images, that is, a parallel processing method is adopted to obtain the position information and rotation information of the target object in each to-be-rendered image; specifically, using Unity's dynamic batching rules, during the game runtime, multiple to-be-rendered images that need to be rendered are stitched together to obtain a stitched atlas (equivalent to stitching multiple to-be-rendered images into a large image, and then directly processing the large image, that is, processing the to-be-rendered images in different regions of the large image, so as to achieve the parallel processing process), and then this atlas is placed under the same canvas, and the shadow images of this atlas are rendered. In this way, Unity will automatically process the to-be-rendered images in different regions of the atlas into a rendering batch and pass it to the GPU to achieve parallel processing.

[0067] Thus, firstly, through the parallel computing ability of the GPU, shadows are generated and updated in real time during the movement of a large number of image elements, i.e., the target object. This solves the problem that the traditional lighting system relies heavily on shadow simulation and cannot efficiently handle the dynamic changes of large-scale images. That is to say, it can calculate the changes of large-scale dynamic shadows in real time without the participation of the traditional lighting system. This not only significantly reduces the consumption of computing resources and improves the rendering efficiency, but also effectively avoids computing bottlenecks during the rendering of large-scale image elements (such as multiple moving objects on the same screen), ensuring the smoothness of real-time rendering. The shadow calculation of each object can be independent and simultaneous without interference with each other, thus improving the rendering throughput. Secondly, by using the batching mechanism of the Unity engine, when rendering a large number of similar objects, such as multiple similar images or objects, they can be placed under the same canvas layer, that is, they can be combined into a single rendering batch. This way of batch merging reduces the state switching during rendering and further improves the rendering efficiency. Especially in the case of limited hardware performance, it can significantly enhance the performance.

[0068] Step 103: Based on the position information, determine the illumination direction of the virtual light source in each image to be rendered, and based on the rotation information, determine the shadow offset in each image to be rendered.

[0069] In actual implementation, the process of determining the illumination direction of the virtual light source in each image to be rendered based on the position information can be as follows: for each image to be rendered, perform the following processing: in the camera coordinate system, obtain the position information of the virtual light source; based on the position information of the virtual light source and the position information of the target object, use the direction in which the virtual light source points to the target object as the illumination direction of the virtual light source in the image to be rendered.

[0070] It should be noted that first, in the world coordinate system, obtain the position information of the virtual light source and the position information of the target object, and transform the position information of the virtual light source and the position information of the target object from the world coordinate system to the camera coordinate system. Thus, in the camera coordinate system, the position information of the virtual light source is obtained, and the position information of the target object is also obtained.

[0071] Among them, the world coordinate system is used to define the positions and orientations of all objects in a three-dimensional scene. It is a global reference framework that provides a common positioning benchmark for all objects in the three-dimensional scene. The world coordinate system is a right-handed coordinate system, usually composed of the following key elements: the origin, the z-axis (vertical axis), the x-axis (horizontal axis), and the y-axis (longitudinal axis). Among them, the origin is the reference point in the world coordinate system, and its coordinates are (0, 0, 0). The positions of all objects are defined relative to this origin; the x-axis is a horizontal axis, usually pointing to the right or left (the specific direction depends on the definition, but usually to the right), and it is used to represent the position of an object in the horizontal direction; the y-axis is also a horizontal axis, usually pointing forward or backward (the specific direction depends on the definition, but usually forward), and it is used to represent the position of an object in the front-back direction; the z-axis is the vertical axis, usually pointing upward. It is used to represent the position of an object in the vertical direction.

[0072] It should be noted that, as mentioned above, the position information is equivalent to coordinates. Therefore, the process of converting the position information of the virtual light source and the position information of the target object from the world coordinate system to the camera coordinate system can be as follows: obtain the view matrix, which defines the position and orientation (i.e., the viewing angle) of the virtual camera. Then, based on the view matrix, convert the coordinates of the virtual light source and the target object in the world coordinate system to the camera coordinate system to obtain the coordinates of the virtual light source and the target object in the camera coordinate system, that is, the position information of the virtual light source and the target object in the camera coordinate system.

[0073] Then, the process of taking the direction from the virtual light source to the target object as the illumination direction of the virtual light source in the to-be-rendered image based on the position information of the virtual light source and the position information of the target object can be as follows: based on the position information of the virtual light source and the position information of the target object, determine the coordinates of the virtual light source and the target object; subtract the coordinates of the target object from the coordinates of the virtual light source to obtain the vector from the virtual light source to the target object, and determine the direction indicated by this vector as the illumination direction of the virtual light source in the to-be-rendered image.

[0074] In actual implementation, the process of determining the shadow offset in each to-be-rendered image based on the rotation information can be as follows: for each to-be-rendered image, perform the following processing to obtain the shadow offset in the to-be-rendered image: based on the rotation information, determine the rotation angle of the target object in the to-be-rendered image; obtain the cosine value and sine value corresponding to the rotation angle, and based on the cosine value and sine value, determine the coordinate offset; multiply the coordinate offset by the pre-set shadow length to obtain the shadow offset, that is:

[0075] y = (cos(θ), sin(θ)) * shadowLength... Formula (1);

[0076] Wherein, y is the shadow offset, (cos(θ), sin(θ)) is the coordinate offset, shadowLength is the shadow length, and θ is the rotation angle, that is, the angle between the irradiation direction of the virtual light source and the horizontal plane (i.e., the x-axis).

[0077] It should be noted that during the movement of the target object, in each frame, its own position, rotation and other information will be transmitted to the shader. After obtaining the cosine value and sine value corresponding to the rotation angle based on the rotation information, the cosine value is determined as the abscissa and the sine value is determined as the ordinate, thereby determining the coordinate offset (cos(θ), sin(θ)). The preset shadow length is a preset constant, and here, the embodiments of the present application do not make any limitations.

[0078] Step 104, for each image to be rendered, based on the irradiation direction, determine the first shadow of the target object in the image to be rendered, and based on the shadow offset and the first shadow, determine the second shadow.

[0079] In actual implementation, for each image to be rendered, for the process of determining the first shadow of the target object in the image to be rendered based on the irradiation direction, refer to Figure 4 , Figure 4 which is a schematic flowchart of determining the first shadow provided by the embodiments of the present application. Based on Figure 4 , the process of determining the first shadow of the target object in the image to be rendered based on the irradiation direction can be implemented through the following steps.

[0080] Step 1041, obtain the unit vector in the irradiation direction and determine multiple vertices corresponding to the target object.

[0081] It should be noted that in a three-dimensional scene, each scene element, that is, an object, corresponds to multiple vertices. A vertex is the basis for constructing a three-dimensional object model, that is, the basic unit that constitutes the surface of the object, and is used to represent the geometric shape and surface properties of the object. Among them, each vertex has a three-dimensional coordinate (x, y, z), and these coordinates determine the position of the vertex in space. By connecting the vertices, triangles or quadrilaterals can be formed, and these polygons are combined to form the surface of the object. Among them, the vertex data in the image to be rendered can be directly obtained in the shader.

[0082] Step 1042, based on the unit vector, select the first vertex in the shadow area from multiple vertices, and form the first shadow based on the first vertex.

[0083] In actual implementation, the process of selecting the first vertex in the shadow area from multiple vertices based on the unit vector can be as follows: in the camera coordinate system, determine the coordinates of each vertex; for each vertex, perform the following processing to obtain the first vertex: perform a dot product operation on the coordinates of the vertex and the unit vector to obtain a dot product result; when the dot product result is less than or equal to zero, determine the vertex as the first vertex in the shadow area.

[0084] It should be noted that for each vertex of the target object, after performing a dot product operation on the coordinates of the vertex and the unit vector to obtain a dot product result; if the dot product result is less than or equal to zero, it means that the vertex is not under the illumination of the light source, that is, the vertex is in the shadow area; if the dot product result is greater than zero, it means that the vertex is under the illumination of the light source. In this way, by traversing all vertices, the entire shadow area can be determined; specifically, performing a dot product operation on the coordinates of the vertex and the unit vector to obtain a dot product result, that is:

[0085] z = P * D = x P *x D +y P *y D +z P *z D …… Formula (2);

[0086] Among them, P is the vertex, with coordinates (x P , y P , z P ), and the unit vector of the illumination direction of the virtual light source is D = (x D , y D , z D ).

[0087] In this way, by customizing the Shader to modify the underlying rendering logic, bypassing the traditional lighting system of game engines such as Unity, and directly using the GPU for shadow calculation. In this way, by optimizing the Shader code, the computational complexity and the number of renderings are reduced, thereby achieving more efficient dynamic shadow simulation.

[0088] In actual implementation, after determining the first vertex in the shadow area, based on the first vertex, form that is, generate the first shadow, where the first shadow is composed of the first vertex.

[0089] In some embodiments, the process of determining the second shadow based on the shadow offset and the first shadow can be as follows: determine the multiple first vertices included in the first shadow; for each first vertex, obtain the coordinates of the first vertex and sum the coordinates of the first vertex and the shadow offset to obtain a second vertex; based on the second vertices corresponding to each first vertex, form the second shadow.

[0090] It should be noted that the shadow offset is equivalent to the vector by which the shadow extends in the light direction. The process of determining the second shadow based on the shadow offset and the first shadow is to extend or move the first shadow by the shadow offset to obtain the second shadow. Here, extending or moving the first shadow by the shadow offset means extending or moving all the vertices of the shadow area along the light direction by a distance corresponding to the shadow offset. Adding the shadow offset to the position of the object, that is, summing with the coordinates of the first vertex, can obtain the position of the shadow, namely:

[0091] S = P + y = P + (cos(θ), sin(θ)) * shadowLength... Formula (3);

[0092] Where, S is the position of the shadow, P is the position of the target object, which is, in actual calculation, the coordinates of the first vertex on the target object that is in the shadow area. y is the shadow offset in Formula (1), (cos(θ), sin(θ)) is the coordinate offset, shadowLength is the shadow length, and θ is the rotation angle.

[0093] For example, assume that the coordinates of the first vertex on the target object that is in the shadow area are (0, 0, 0), and the light source is directly to the right of the object, which means θ is π / 2 (90 degrees). Therefore, (cos(θ), sin(θ)) will be (0, 1). Assume that shadowLength is 10, then the position of the shadow, that is, the second vertex corresponding to the respective first vertex, S = (0, 0, 0) + (cos(θ), sin(θ)) * shadowLength = (0, 0, 0) + (0, 1) * 10 = (0, 10, 0), which means the shadow will be at a position 10 unit distances directly to the right of the target object.

[0094] It should be noted that the first shadow is the shadow area on the target object that is not in the illuminated area, and the second shadow is the shadow area corresponding to the target object formed when the virtual light source irradiates the target object. For example, when the target object is on the ground, the second shadow is the shadow area formed on the ground when the virtual light source irradiates the target object.

[0095] Step 105: Render the multiple images to be rendered based on each second shadow to obtain multiple shadow images.

[0096] In actual implementation, the process of rendering multiple images to be rendered based on each second shadow to obtain multiple shadow images may be as follows: for each image to be rendered, obtain the texture image corresponding to the image to be rendered; based on the texture image, determine the shadow intensity of the image pixel points in the second shadow in the image to be rendered, and the color values of the image pixel points outside the second shadow in the image to be rendered; based on the shadow intensity of the image pixel points in the second shadow in the image to be rendered and the color values of the image pixel points outside the second shadow, render the corresponding shadow image.

[0097] It should be noted that the process of obtaining the texture image corresponding to the image to be rendered may be as follows: obtain the attribute information (including position, size, shape, etc.) of the virtual object in the image to be rendered in the model space corresponding to the virtual scene; based on the attribute information of the virtual object, determine at least two vertices of the virtual object in the model space, construct the spatial coordinate system corresponding to the model space, and in the spatial coordinate system, determine the coordinates and attributes (including normal, texture coordinate, tangent, etc.) of each vertex; perform coordinate transformation on the coordinates of each vertex in the spatial coordinate system to obtain the coordinates of each vertex in the screen coordinate system; use the coordinates of each vertex in the screen coordinate system and the attributes of each vertex as the texture information of the first virtual object.

[0098] Among them, the screen coordinate system is a two-dimensional reference system for describing the position of a touch point (such as a cursor) on a physical device (such as a display); among them, the origin of the screen coordinate system is located at the upper left corner of the screen, the X-axis is positive to the right, the Y-axis is positive downward, and the measurement unit is pixels, which is used to locate points on the screen corresponding to the terminal, such as the position of a mouse click, the position of a window, etc.

[0099] Then, perform rasterization based on the texture information of the virtual object to obtain the texture image corresponding to the image to be rendered, that is, the uncolored two-dimensional image of the image to be rendered in the screen space; among them, the rasterization here is used to convert the geometric data of the virtual object into pixel data, that is, to convert vector graphics (such as straight lines and curves) into raster images composed of pixels.

[0100] In actual implementation, the process of determining the shadow intensity of the image pixel points in the second shadow in the image to be rendered and the color values of the image pixel points outside the second shadow in the image to be rendered based on the texture image may be as follows: for each image pixel point in the texture image, perform lighting calculation (that is, lighting rendering, shading) based on the texture information of the virtual object to obtain the shadow intensity of the image pixel points in the second shadow in the image to be rendered, and the color values of the image pixel points outside the second shadow in the image to be rendered.

[0101] Then, based on the shadow intensity of the image pixel points in the second shadow and the color values of the image pixel points outside the second shadow in the to-be-rendered image, a corresponding shadow image is rendered; in this way, multiple shadow images are obtained.

[0102] In actual implementation, when rendering multiple to-be-rendered images to obtain multiple shadow images, the shadow rendering process can also be optimized, so as to obtain multiple shadow images with better shadow rendering effects. Next, taking three optimization methods as examples respectively, the process of optimizing the shadow rendering process will be described.

[0103] In some embodiments, after determining the second shadow based on the shadow offset and the first shadow, for each to-be-rendered image, the following processing can also be performed: performing convolution processing on the to-be-rendered image to obtain the horizontal gradient of the to-be-rendered image in the horizontal direction and the vertical gradient in the vertical direction; based on the horizontal gradient and the vertical gradient, determining the edge region in the second shadow, and performing soft processing on the edge region in the second shadow to obtain the third shadow; thus, the process of rendering multiple to-be-rendered images based on each second shadow to obtain multiple shadow images can be to render multiple to-be-rendered images based on each third shadow to obtain multiple shadow images.

[0104] It should be noted that since gradient calculation is usually performed on grayscale images, it is necessary to first convert the to-be-rendered image into a grayscale image, so as to perform convolution processing on the to-be-rendered image to obtain the horizontal gradient of the to-be-rendered image in the horizontal direction and the vertical gradient in the vertical direction. Specifically, the process includes performing grayscale processing on the to-be-rendered image to obtain a grayscale image; then performing convolution processing on the grayscale image to obtain the horizontal gradient of the grayscale image in the horizontal direction and the vertical gradient in the vertical direction;

[0105] It should be noted that in the shader, the edges of the picture can be detected, and then mapped to the edges of the shadow, that is, the area that needs to be softened, and then the softness processing, that is, blurring processing, is performed on this area to make the shadow edge softer and more natural, reducing the unrealistic feeling brought by hard shadows.

[0106] In actual implementation, first, two 3x3 convolution kernels are obtained, which are respectively used to detect the edges in the horizontal and vertical directions. Then, based on these two convolution kernels, the grayscale image is convolved respectively, and the brightness gradients of the grayscale image in the horizontal and vertical directions can be obtained. Specifically, based on the horizontal convolution kernel and the vertical convolution kernel, the grayscale image is convolved to obtain the horizontal gradient and the vertical gradient, that is:

[0107] I x =I*G x ……Formula (4);

[0108] I y = I * G y ……Formula (5);

[0109] Among them, I in Formula (4) x is the horizontal gradient, and G x is the convolution kernel for detecting edges in the horizontal direction. I in Formula (5) y is the vertical gradient, and G y is the convolution kernel for detecting edges in the vertical direction. I is the grayscale image corresponding to the image to be rendered.

[0110] It should be noted that for the convolution processing of the grayscale image based on the horizontal convolution kernel and the vertical convolution kernel to obtain the horizontal gradient and the vertical gradient, what is obtained is the horizontal gradient and the vertical gradient of each pixel point in the grayscale image.

[0111] In actual implementation, the process of determining the edge region in the second shadow based on the horizontal gradient and the vertical gradient can be as follows: for each pixel point in the grayscale image, sum the square of the horizontal gradient of the pixel point and the square of the vertical gradient of the pixel point to obtain a summation result; perform a square root operation on the summation result to obtain the edge intensity of the pixel point, that is:

[0112]

[0113] Among them, E is the edge intensity of the pixel point, I x is the horizontal gradient, and I y is the vertical gradient.

[0114] Then, determine the target pixel points with edge intensity greater than the edge intensity threshold from multiple pixel points, and take the region where the target pixel points are located as the edge region in the image to be rendered; among them, the edge intensity threshold is preset, and in this regard, the embodiments of the present application do not make any limitations.

[0115] In actual implementation, after determining the edge region in the image to be rendered, the process of softening the edge region in the second shadow to obtain the third shadow can be to perform blurring processing on the edge region, such as Gaussian blurring processing, to obtain the blurred second shadow, that is, the third shadow.

[0116] Then, based on each third shadow, render multiple images to be rendered to obtain multiple shadow images. Here, the process of rendering multiple images to be rendered based on each third shadow to obtain multiple shadow images is similar to the process described above of rendering multiple images to be rendered based on each second shadow, and in this regard, the embodiments of the present application will not elaborate.

[0117] Thus, after detecting the edge region of the image to be rendered, by blurring the edge region, the shadow region at the edge can be made more natural and soft, reducing the unrealistic feeling brought by hard shadows.

[0118] In some embodiments, after determining the second shadow based on the shadow offset and the first shadow, for each image to be rendered, the following processing may further be performed: obtaining the distance between the target object and the virtual light source and the mapping relationship, where the mapping relationship is used to indicate the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object; determining the target shadow adjustment coefficient corresponding to the target object based on the mapping relationship and the distance between the target object and the virtual light source; determining the shadow intensity of the second shadow, and adjusting the shadow intensity of the second shadow based on the target shadow adjustment coefficient to obtain the fourth shadow; thus, the process of rendering multiple images to be rendered based on each second shadow to obtain multiple shadow images may be to render multiple images to be rendered based on each fourth shadow to obtain multiple shadow images.

[0119] It should be noted that the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object, that is, the mapping relationship, is preset. Here, the shadow adjustment coefficient is used to adjust the shadow intensity of the shadow region corresponding to the corresponding object. Thus, based on the shadow adjustment coefficient, the shadow intensities of the shadow regions of different objects are adjusted. If the object is relatively close to the virtual light source, the shadow intensity of the shadow region of the object is enhanced based on the corresponding shadow adjustment coefficient. If the object is relatively far from the virtual light source, the shadow intensity of the shadow region of the object is weakened based on the corresponding shadow adjustment coefficient.

[0120] It should be noted that the process of rendering multiple images to be rendered based on each fourth shadow to obtain multiple shadow images is similar to the process of rendering multiple images to be rendered based on each second shadow described above. In this regard, the embodiments of the present application will not be elaborated.

[0121] Thus, according to the distances between different objects and the light source, shadow intensities with different levels of layering are simulated, making the shadows closer to the light source darker and the shadows farther away gradually fade, creating a more realistic shadow effect.

[0122] In some other embodiments, the number of target objects is multiple. After determining the second shadow based on the shadow offset and the first shadow, the following processing may further be performed for each to-be-rendered image: Select target object pairs from the multiple target objects; wherein, the two target objects included in the target object pair have an overlapping shadow area; for each target object pair, determine the shadow intensity of the overlapping shadow area of each target object in the target object pair, and select the larger shadow intensity from the two shadow intensities; based on the larger shadow intensity, update the shadow intensity of the overlapping shadow area of each target object in the target object pair to obtain the fifth shadow of each target object in the target object pair; and the process of rendering the multiple to-be-rendered images based on the respective second shadows to obtain multiple shadow images may be to render the multiple to-be-rendered images based on the fifth shadow of each target object in the target object pair and the second shadows of other target objects to obtain multiple shadow images; wherein, the other target objects are the target objects other than the target objects in the target object pair among the multiple target objects.

[0123] It should be noted that if the to-be-rendered image includes multiple target objects, two target objects with overlapping corresponding shadow areas are determined as the target object pair; wherein, the number of the target object pairs here may be one or more, and the embodiments of the present application do not make any limitation thereto; specifically, the multiple target objects are combined in pairs to obtain multiple object combinations; from the multiple object combinations, select the object combination in which the second shadows corresponding to the two target objects included therein have an overlapping area, that is, select the object combination in which the two target objects included therein have an overlapping shadow area as the target object pair.

[0124] Then, for each target object pair, obtain the shadow intensity of the overlapping shadow area of the two target objects included in the target object pair; then compare the two shadow intensities, select the larger shadow intensity from the two shadow intensities, and use the larger shadow intensity as the shadow intensity of the overlapping shadow area of the corresponding two target objects, and based on the larger shadow intensity, update the shadow intensity of the overlapping shadow area of each target object in the target object pair to obtain the fifth shadow of each target object in the target object pair.

[0125] It should be noted that the process of rendering the multiple to-be-rendered images based on the fifth shadow of each target object in the target object pair and the second shadows of other target objects to obtain multiple shadow images is similar to the process of rendering the multiple to-be-rendered images based on the respective second shadows to obtain multiple shadow images described above, and the embodiments of the present application do not elaborate thereon.

[0126] In this way, when there are a large number of objects moving at the same time in the scene, there may be many objects very close to each other. Their shadows superimposed on each other will look very stiff and will produce a certain amount of overdraw, causing performance loss. In this way, through the characteristic that the shadows of objects farther away from the assumed light source will be lighter, if two objects have overlapping shadow areas, only the darker shadow will be displayed, avoiding the problem of stiff display effect caused by shadow superposition and reducing performance consumption.

[0127] It should be noted that, when actually optimizing the shadow rendering process, at least one of the above-mentioned optimization methods may be selected to optimize the shadow rendering process, and this embodiment of the present application does not limit this.

[0128] In some embodiments, before optimizing the shadow rendering process of the target object, the size of the target object may be detected. If the detection result indicates that the size of the target object is larger than the target size, at least one of the above-mentioned optimization methods is selected to optimize the shadow rendering process of the corresponding target object. If the size of the target object is less than or equal to the target size, the corresponding target object may not be optimized because these small target objects will not affect the user experience. The target size here is pre-set and is not limited in the embodiments of the present application. In this way, the rendering efficiency and game performance may be further improved.

[0129] In some embodiments, the above embodiments of the present application can also be implemented through similar API interface calls, that is, the above embodiments can be ported to other engines that support custom Shaders to provide consistent rendering effects. In this way, the above embodiments of the present application can be used on multiple rendering platforms to ensure cross-platform compatibility.

[0130] By applying the above-mentioned embodiment of the present application, after a plurality of images to be rendered including a dynamic target object are captured based on a virtual camera, the position information and rotation information of the target object in each image to be rendered are obtained, and then the illumination direction of the virtual light source in each image to be rendered is determined based on the position information, and the shadow offset is determined in each image to be rendered based on the rotation information, thereby determining the second shadow based on the shadow offset and the first shadow, and then rendering the plurality of images to be rendered based on each second shadow to obtain a plurality of shadow images. In this way, compared with the solution in the related art in which the rendered shadow is inconsistent with the real shadow if the object is rotated, the present application determines the shadow offset in each image to be rendered through the rotation information, and then determines the second shadow to be rendered based on the shadow offset and the first shadow of the target object determined based on the illumination direction, thereby ensuring the rendering effect of the shadow image; at the same time, the rendering process does not rely on complex illumination models and calculations, and also improves the rendering efficiency.

[0131] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0132] In the related art, common dynamic shadow calculation methods include Shadow Mapping and Ray Traced Shadows based on the lighting system. These techniques calculate shadows by simulating the light propagation and occlusion relationship between the light source and the object. However, these methods usually rely on relatively complex lighting models and calculations, especially in scenes with a large number of moving objects or complex scenes, where the rendering cost is very high. Therefore, the above-mentioned techniques are generally used for shadow rendering in large 3D scene projects. In addition, the lighting systems provided by game engines such as Unity can also implement shadow calculations, but these systems usually rely on the Central Processing Unit (CPU) and Graphics Processing Unit (GPU) for a large amount of lighting and shadow calculations. When rendering a large number of moving images on the same screen, performance bottlenecks are likely to occur. Existing technologies often cannot maintain good image quality and smoothness under low hardware configurations, especially on low-end mobile phones or devices.

[0133] Based on this, the shadow rendering methods for moving objects in the related art have the following problems:

[0134] First, performance bottleneck: The shadow calculation methods in the related art (such as shadow mapping and ray tracing) will cause a sharp drop in performance when rendering scenes with high dynamic range and large-scale objects. Especially on low-end devices, it is difficult to meet the requirements of real-time rendering. Especially when there are a large number of moving objects on the screen, the complexity and performance consumption of shadow calculation are very high.

[0135] Second, high computational resource consumption: The shadow calculation methods in the related art usually rely on relatively high computational resources. Especially for ray tracing technology, it requires a large amount of calculation time and GPU resources, and cannot adapt to the real-time requirements of low-end hardware devices, resulting in inconsistent performance of games or applications on different performance devices and affecting the user experience.

[0136] Third, dependence on the lighting system: Most of the related technologies rely on traditional lighting systems to simulate shadows, and this method limits the real-time update and processing efficiency of shadows. Especially during the dynamic shadow rendering process of large-scale moving objects, the calculation of the lighting system often becomes a performance bottleneck.

[0137] Fourth, too many rendering batches or high OverDraw: The shadow component provided by the Unity engine is achieved by rendering twice with dislocation, but this method will result in a very high proportion of overlapping rendering areas in the picture, becoming a performance bottleneck during large-scale object movement.

[0138] Fifth, in some existing simulation schemes that do not rely on lighting (such as the Shadow component provided by the development engine), the problem that the shadow needs to always maintain a fixed direction relative to the assumed light source during the rotation of the object has not been solved. When the object rotates, the shadow will rotate along with it, resulting in inconsistent performance with real shadows.

[0139] Based on this, the present application provides a shadow rendering method. Through custom Shader and parallel computing, it aims to simulate the effect of dynamic shadows during the movement of large-scale 2D images on the same screen, and does not rely on Unity's lighting system, avoiding the dependence on the traditional lighting system. At the same time, by adopting an efficient dynamic shadow simulation algorithm, it solves the problems of poor dynamic shadow calculation performance, high resource consumption, and excessive hardware requirements in the prior art, enabling the rendering process of dynamic shadows to be completed with relatively low computational resource consumption and ensuring the image quality in real-time rendering, so that users using mid- to low-end mobile phones can also have a good experience. The specific solutions are as follows:

[0140] 1. By deeply optimizing the Shader and utilizing the efficient computing power of the GPU, the changes in shadows are dynamically calculated instead of relying on traditional lighting models, reducing the computational burden during the rendering process, solving the problem of inconsistent shadow performance after image rotation, maintaining high rendering quality, and being able to efficiently handle the shadow calculations of a large number of moving images, avoiding the performance bottleneck caused by a large amount of calculations in traditional technologies, thereby significantly reducing the consumption of computational resources and improving the rendering efficiency;

[0141] 2. Through the parallel computing power of the GPU, it is possible to achieve maintaining a single rendering batch when rendering different states of a large number of similar objects, improving the rendering performance. Especially in high-load scenarios, the shadow calculations of a large number of moving images can be processed in parallel, greatly increasing the throughput of the system;

[0142] 3. It can not only run efficiently in the Unity engine but also be applicable to other game engines or rendering systems, with strong versatility and scalability, enabling users to still obtain good dynamic shadow effects and smooth game experiences even on devices with relatively weak hardware performance.

[0143] In actual implementation, the present application utilizes the parallel computing power of the GPU to optimize the shadow rendering process through custom Shader, breaking through the limitations of the traditional lighting system and solving the problems of low efficiency and high resource consumption in dynamic shadow rendering in related technologies. The following is the detailed technical implementation process:

[0144] First, simulate shadows using object geometry data

[0145] Based on the outline of the image (the shader can obtain the vertex data of the image) and the assumed light source direction, vector operations and dot products are used to determine which areas are in the shadow area of the image. Specifically, vector operations and dot products can be used to calculate whether each vertex of the object is illuminated by the light source. Let the vertex of the object be P = (x P , y P , z P ), and the light source direction be D = (x D , y D , z D ), where D is a unit vector, and the dot product formula is as shown in formula (2) above. Thus, if P * D ≤ 0, then the vertex is in the shadow area. By traversing all vertices, the entire shadow area can be determined.

[0146] Second, shadow offset based on object movement

[0147] During the movement of the object, in each frame, its position, rotation, and other information are passed to the shader. According to the following formula, the position of the shadow can be obtained, that is:

[0148] S = y + (cos(θ), sin(θ)) * shadowLength... Formula (7);

[0149] where y is the shadow area on the object determined in the first step, (cos(θ), sin(θ)) is the coordinate offset, shadowLength is the preset shadow length, and θ is the rotation angle, that is, the angle between the illumination direction of the virtual light source and the horizontal plane (i.e., the x-axis).

[0150] Third, adjust the softness of the shadow;

[0151] In actual implementation, in the shader, the edges of the image can be detected, and then mapped to the edges of the shadow, that is, the area that needs to be softened. Then, softness processing, that is, blurring processing, is performed on this area to make the shadow edge softer and more natural, reducing the unrealistic feeling brought by hard shadows.

[0152] Among them, to detect the edge by calculating the spatial gradient of the image brightness, first, two 3x3 convolution kernels are obtained, which are used to detect the edges in the horizontal and vertical directions respectively. Then, based on these two convolution kernels, the grayscale image corresponding to the image to be rendered is convolved respectively, and the brightness gradients in the horizontal and vertical directions of the grayscale image can be obtained. Specifically, based on the horizontal convolution kernel and the vertical convolution kernel, the grayscale image corresponding to the image to be rendered is convolved to obtain the horizontal gradient and the vertical gradient, as shown in formula (4) and formula (5) above.

[0153] Then, the edge strength of each pixel is determined based on the above formula (6), thereby determining the target pixel whose edge strength is greater than the edge strength threshold from multiple pixels, and the area where the target pixel is located is used as the edge area in the image to be rendered; wherein the edge strength threshold is pre-set, and this embodiment of the present application does not limit this. Then, after determining the edge area, the edge area is subjected to softness processing, i.e., blur processing, such as Gaussian blur processing.

[0154] Fourth, multi-layered shadow simulation

[0155] According to the distance between different objects and the light source, different levels of shadow intensity are simulated, making the shadows near darker and the shadows far away gradually lighter, creating a more realistic shadow effect.

[0156] Fifth, merge superimposed shadows

[0157] When a large number of objects are moving at the same time in the scene, there may be many objects that are very close to each other. Their shadows superimposed on each other will look very stiff and will produce a certain amount of overdraw, resulting in performance loss. In this way, through the characteristic that the shadows of objects farther away from the assumed light source will be lighter, if two objects have overlapping shadow areas, only the darker shadow will be displayed, avoiding the problem of stiff display effect caused by shadow superposition and reducing performance consumption.

[0158] Sixth, optimize rendering performance

[0159] Using Unity's dynamic batching rules, multiple images to be rendered are stitched together when the game is running to obtain a stitched atlas (equivalent to stitching multiple images to be rendered into a large image, so that the large image can be directly processed, that is, the images to be rendered in different areas of the large image are processed, thereby realizing a parallel processing process), and then this atlas is placed under the same canvas, and the shadow image of this atlas is rendered. In this way, Unity will automatically process the images to be rendered in different areas of the atlas into a rendering batch and pass it to the GPU to realize parallel processing.

[0160] In addition, for some parameters that need to be passed into the shader (such as the position of objects and light sources, the rotation of objects, and other information), do not modify the Property module provided by the shader, because this will cause Unity batching to fail. Instead, you can use the default color attributes of the game object to pass the parameters. For example, the color consists of four values ​​(r, g, b, a). The parameters that need to be passed are simplified and encrypted. For example, the position information (156, 234) is converted into 0.156 and 0.234 and assigned to the color attributes r and g respectively. Then, the obtained color attributes are decrypted into real information in the shader.

[0161] Seventh, LOD (Level of Detail) optimization

[0162] In actual implementation, the size of an object (target object) can also be detected. If the detection result indicates that the size of the target object is greater than the target size, the processes of steps two, three, and four above are selected to process the corresponding target object. If the size of the target object is less than or equal to the target size, since these small target objects will not affect the user experience, the processes of steps two, three, and four above do not need to be selected for the corresponding target object. In this way, the rendering efficiency and game performance can be further improved.

[0163] Seventh, cross-platform adaptation

[0164] The above embodiments of the present application can also be implemented by calling through a similar API interface, that is, the above embodiments can be transplanted into other engines that support custom Shaders to provide a consistent rendering effect. In this way, the above embodiments of the present application can be universal on multiple rendering platforms, ensuring cross-platform compatibility.

[0165] In this way, through the above embodiments, the custom Shader can effectively simulate the dynamic shadow effect without relying on real-time lighting, while maintaining high performance and a good user experience. These methods combine the geometric data, motion information, and screen space technology of the object to create a more realistic and optimized shadow effect. Specifically, the custom Shader and GPU parallel computing model of the present application achieve deep optimization of the shadow calculation process, effectively reducing the consumption of computing resources, especially reducing the dependence on the CPU, so that efficient real-time rendering can be maintained even in high-load image scenarios. At the same time, the rendering batch merging strategy improves the rendering efficiency while reducing the resource requirements of the system, ensuring performance optimization. Moreover, the introduced adaptive rendering strategy can automatically adjust the shadow quality according to the device performance, enabling smooth operation on devices with lower performance and ensuring the consistency of the user experience. In addition, the core algorithm and Shader optimization design of the present application are independent of engines such as Unity, support cross-platform rendering applications, and can be applied to other mainstream graphics engines, and also have wide universality and scalability.

[0166] In actual implementation, based on the present application, there are also some extended or alternative ways to increase its flexibility and applicability. For example:

[0167] 1. More flexible shadow generation method: The real-time rendering of dynamic shadows is completed through parallel computing. An "approximate shadow" generation method based on the preset motion trajectory of the object can be considered, especially in scenarios where the shadow accuracy requirement is relatively low. This method predicts and generates shadows by calculating the overall trend of the object's motion, which can further reduce the GPU occupancy.

[0168] 2. Expansion of applicable data types: Although the main description object of this application is the dynamic shadow rendering of moving objects in images, the same technical process and rendering concept can be applied to the processing of other types of multimedia data, such as the dynamic light and shadow simulation in video frames, so as to enhance the versatility of rendering.

[0169] 3. Cross-platform interface optimization: This application can be further refined to adapt to the interfaces of more rendering engines. For example, develop versions specifically adapted to Unreal or other 3D engines, improve the compatibility between engines by optimizing the cross-platform rendering interface, and provide a wider range of applications on different devices.

[0170] 4. Compatibility with 2D and 3D rendering: By further expanding the parallel computing model to simulate the dynamic shadows of 3D objects and merge them with 2D images, it is convenient for rendering in 3D / 2D hybrid scenarios and enhances the compatibility of the solution.

[0171] Applying the above embodiments of the present application, after collecting multiple images to be rendered including dynamic target objects based on a virtual camera, obtaining the position information and rotation information of the target object in each image to be rendered, then determining the irradiation direction of the virtual light source in each image to be rendered based on the position information, and determining the shadow offset in each image to be rendered based on the rotation information. Thus, based on the shadow offset and the first shadow, the second shadow is determined, and then based on each second shadow, multiple images to be rendered are rendered to obtain multiple shadow images. In this way, compared with the related art where the rendered shadow and the real shadow performance are inconsistent when the object rotates, the present application determines the shadow offset in each image to be rendered through the rotation information, and then determines the second shadow to be rendered based on the shadow offset and the first shadow of the target object determined based on the irradiation direction, ensuring the rendering effect of the shadow image; at the same time, the rendering process does not rely on complex lighting models and calculations, and also improves the rendering efficiency.

[0172] Next, continue to illustrate the exemplary structure of the software module implementation of the shadow image rendering device 455 provided by the embodiments of the present application. In some embodiments, as Figure 2 shown, the software module stored in the shadow image rendering device 455 in the memory 450 may include:

[0173] The acquisition module 4551 is used to acquire a plurality of to-be-rendered images including the target object based on a virtual camera; wherein, the to-be-rendered images include the shadow of the target object, and in the image frame sequence formed by the plurality of to-be-rendered images, the position of the target object changes continuously;

[0174] The acquisition module 4552 is used to acquire the position information and rotation information of the target object in each of the to-be-rendered images in the camera coordinate system;

[0175] The first determination module 4553 is used to determine the illumination direction of the virtual light source in each of the to-be-rendered images based on the position information, and determine the shadow offset in each of the to-be-rendered images based on the rotation information;

[0176] The second determination module 4554 is used to, for each of the to-be-rendered images, determine the first shadow of the target object in the to-be-rendered image based on the illumination direction, and determine the second shadow based on the shadow offset and the first shadow;

[0177] The rendering module 4555 is used to render the plurality of to-be-rendered images based on each of the second shadows to obtain a plurality of shadow images.

[0178] In some embodiments, the first determination module 4553 is further used to, for each of the to-be-rendered images, perform the following processing: acquire the position information of the virtual light source in the camera coordinate system; based on the position information of the virtual light source and the position information of the target object, use the direction in which the virtual light source points to the target object as the illumination direction of the virtual light source in the to-be-rendered image.

[0179] In some embodiments, the first determination module 4553 is further used to, for each of the to-be-rendered images, perform the following processing to obtain the shadow offset in the to-be-rendered image: determine the rotation angle of the target object in the to-be-rendered image based on the rotation information; acquire the cosine value and sine value corresponding to the rotation angle, and determine the coordinate offset based on the cosine value and the sine value; perform a multiplication process on the coordinate offset and a preset shadow length to obtain the shadow offset.

[0180] In some embodiments, the second determination module 4554 is further used to acquire the unit vector in the illumination direction and determine a plurality of vertices corresponding to the target object; based on the unit vector, select the first vertex in the shadow area from the plurality of vertices, and form the first shadow based on the first vertex.

[0181] In some embodiments, the second determination module 4554 is further configured to determine the coordinates of each of the vertices in the camera coordinate system; for each of the vertices, perform the following processing to obtain the first vertex: perform a dot product operation on the coordinates of the vertex and the unit vector to obtain a dot product result; when the dot product result is less than or equal to zero, determine the vertex as the first vertex in the shadow region.

[0182] In some embodiments, the second determination module 4554 is further configured to determine a plurality of first vertices included in the first shadow; for each of the first vertices, obtain the coordinates of the first vertex, and sum the coordinates of the first vertex and the shadow offset to obtain a second vertex; based on the second vertices corresponding to the respective first vertices, form the second shadow.

[0183] In some embodiments, the apparatus further includes a first shadow adjustment module, and the first shadow adjustment module is configured to perform the following processing for each of the to-be-rendered images: perform a convolution process on the to-be-rendered image to obtain a horizontal gradient of the to-be-rendered image in the horizontal direction and a vertical gradient in the vertical direction; based on the horizontal gradient and the vertical gradient, determine an edge region in the second shadow, and perform a softening process on the edge region in the second shadow to obtain a third shadow; the rendering module 4555 is further configured to render the plurality of to-be-rendered images based on the respective third shadows to obtain a plurality of shadow images.

[0184] In some embodiments, the apparatus further includes a second shadow adjustment module, and the second shadow adjustment module is configured to perform the following processing for each of the to-be-rendered images: obtain the distance and the mapping relationship between the target object and the virtual light source, where the mapping relationship is used to indicate the correspondence between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object; based on the mapping relationship and the distance between the target object and the virtual light source, determine the target shadow adjustment coefficient corresponding to the target object; determine the shadow intensity of the second shadow, and adjust the shadow intensity of the second shadow based on the target shadow adjustment coefficient to obtain a fourth shadow; the rendering module 4555 is further configured to render the plurality of to-be-rendered images based on the respective fourth shadows to obtain a plurality of shadow images.

[0185] In some embodiments, the device further includes a third shadow adjustment module, which is configured to perform the following processing on each of the to-be-rendered images: select a pair of target objects from the multiple target objects, where the pair of target objects includes two target objects with an overlapping shadow area; for each pair of target objects, determine the shadow intensity of the overlapping shadow area of each target object in the pair of target objects, and select the larger shadow intensity from the two shadow intensities; based on the larger shadow intensity, update the shadow intensity of the overlapping shadow area of each target object in the pair of target objects to obtain the fifth shadow of each target object in the pair of target objects; the rendering module 4555 is further configured to render the multiple to-be-rendered images based on the fifth shadow of each target object in the pair of target objects and the second shadow of other target objects to obtain multiple shadow images, where the other target objects are the target objects other than the target objects in the pair of target objects among the multiple target objects.

[0186] An embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for rendering a shadow image or the job matching method described above in the embodiments of the present application. For example, as Figure 3 the method for rendering a shadow image shown.

[0187] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, where the executable instructions, when executed by a processor, cause the processor to execute the method for rendering a shadow image or the job matching method provided in the embodiments of the present application. For example, as Figure 3 the method for rendering a shadow image shown.

[0188] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a CD-ROM, etc.; it may also be various devices including one or any combination of the above memories.

[0189] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a stand-alone program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0190] As an example, the executable instructions may or may not correspond to a file in a file system, may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or, stored in multiple cooperating files (such as files that store one or more modules, subroutines, or portions of code).

[0191] As an example, the executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or, on multiple electronic devices distributed across multiple locations and interconnected by a communication network.

[0192] It should be noted that in the embodiments of the present application, when it comes to obtaining relevant data such as videos and facial scan data including the target object, when the embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0193] The above are only the embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are all included in the protection scope of the present application.

Claims

1. A method for rendering a shadow image, characterized in that: The method comprises: Based on the virtual camera, a plurality of images to be rendered including the target object are collected; The image to be rendered includes a shadow of the target object, and in an image frame sequence formed by the multiple images to be rendered, the position of the target object changes continuously; In the camera coordinate system, obtaining the position information and rotation information of the target object in each of the images to be rendered; Based on the position information, determining the illumination direction of the virtual light source in each of the images to be rendered, and based on the rotation information, determining the shadow offset in each of the images to be rendered; For each of the images to be rendered, determining a first shadow of the target object in the image to be rendered based on the illumination direction, and determining a second shadow based on the shadow offset and the first shadow; Based on each of the second shadows, the multiple images to be rendered are rendered to obtain multiple shadow images.

2. The method according to claim 1, characterized in that The step of determining the illumination direction of the virtual light source in each of the images to be rendered based on the position information includes: For each of the images to be rendered, the following processing is performed: In the camera coordinate system, obtaining position information of the virtual light source; Based on the position information of the virtual light source and the position information of the target object, the virtual light source is pointed in the direction of the target object as the illumination direction of the virtual light source in the image to be rendered.

3. The method according to claim 1, characterized in that Determining the shadow offset in each of the to-be-rendered images based on the rotation information includes: For each of the images to be rendered, the following processing is performed to obtain the shadow offset in the image to be rendered: Based on the rotation information, determining a rotation angle of the target object in the image to be rendered; Obtaining a cosine value and a sine value corresponding to the rotation angle, and determining a coordinate offset based on the cosine value and the sine value; The coordinate offset is multiplied by a preset shadow length to obtain the shadow offset.

4. The method according to claim 1, characterized in that: The determining, based on the illumination direction, a first shadow of the target object in the image to be rendered includes: Obtaining a unit vector in the irradiation direction, and determining a plurality of vertices corresponding to the target object; Based on the unit vector, a first vertex in the shadow area is selected from the multiple vertices, and the first shadow is formed based on the first vertex.

5. The method according to claim 4, characterized in that The step of selecting a first vertex in a shadow area from the plurality of vertices based on the unit vector comprises: In the camera coordinate system, determining the coordinates of each of the vertices; For each of the vertices, perform the following processing to obtain the first vertex: Performing a dot product process on the coordinates of the vertex and the unit vector to obtain a dot product result; When the dot product result is less than or equal to zero, the vertex is determined to be the first vertex in the shadow area.

6. The method according to claim 1, characterized in that The determining the second shadow based on the shadow offset and the first shadow includes: determining a plurality of first vertices included in the first shadow; For each of the first vertices, obtain the coordinates of the first vertex, and sum the coordinates of the first vertex and the shadow offset to obtain a second vertex; The second shadow is formed based on the second vertices corresponding to the first vertices.

7. The method according to claim 1, characterized in that After determining the second shadow based on the shadow offset and the first shadow, the method further includes: For each of the images to be rendered, the following processing is performed: Performing convolution processing on the image to be rendered to obtain a horizontal gradient of the image to be rendered in a horizontal direction and a vertical gradient of the image to be rendered in a vertical direction; Based on the horizontal gradient and the vertical gradient, determine an edge region in the second shadow, and perform a softening process on the edge region in the second shadow to obtain a third shadow; The step of rendering the plurality of images to be rendered based on the second shadows to obtain a plurality of shadow images comprises: Based on each of the third shadows, the multiple images to be rendered are rendered to obtain multiple shadow images.

8. The method according to claim 1, characterized in that After determining the second shadow based on the shadow offset and the first shadow, the method further includes: For each of the images to be rendered, the following processing is performed: Acquire the distance between the target object and the virtual light source and a mapping relationship, wherein the mapping relationship is used to indicate the corresponding relationship between the distance between the object and the virtual light source and the shadow adjustment coefficient corresponding to the object; Determining a target shadow adjustment coefficient corresponding to the target object based on the mapping relationship and the distance between the target object and the virtual light source; determining a shadow intensity of the second shadow, and adjusting the shadow intensity of the second shadow based on the target shadow adjustment coefficient to obtain a fourth shadow; The step of rendering the plurality of images to be rendered based on the second shadows to obtain a plurality of shadow images comprises: Based on each of the fourth shadows, the multiple images to be rendered are rendered to obtain multiple shadow images.

9. The method according to claim 1, characterized in that: The number of the target objects is multiple; after determining the second shadow based on the shadow offset and the first shadow, the method further includes: For each of the images to be rendered, the following processing is performed: Selecting a target object pair from the plurality of target objects; wherein two target objects included in the target object pair have overlapping shadow areas; For each of the target object pairs, determine the shadow intensity of the overlapping shadow area of ​​each target object in the target object pair, and select a larger shadow intensity from the two shadow intensities; based on the larger shadow intensity, update the shadow intensity of the overlapping shadow area of ​​each target object in the target object pair to obtain a fifth shadow of each target object in the target object pair; The step of rendering the plurality of images to be rendered based on the second shadows to obtain a plurality of shadow images comprises: Rendering the plurality of images to be rendered based on the fifth shadow of each target object in the target object pair and the second shadows of other target objects to obtain a plurality of shadow images; The other target objects are target objects other than the target object in the target object pair among the multiple target objects.

10. A shadow image rendering device, characterized in that: The device comprises: A collection module, used for collecting a plurality of images to be rendered including a target object based on a virtual camera; wherein the images to be rendered include a shadow of the target object, and in an image frame sequence formed by the plurality of images to be rendered, a position of the target object changes continuously; An acquisition module, used for acquiring the position information and rotation information of the target object in each of the images to be rendered in a camera coordinate system; A first determination module, configured to determine an illumination direction of a virtual light source in each of the images to be rendered based on the position information, and to determine a shadow offset in each of the images to be rendered based on the rotation information; A second determining module is used to determine, for each of the images to be rendered, a first shadow of the target object in the image to be rendered based on the illumination direction, and to determine a second shadow based on the shadow offset and the first shadow; A rendering module is used to render the multiple images to be rendered based on each of the second shadows to obtain multiple shadow images.

Images