Data processing method and device, storage medium and electronic device

Abstract

The application discloses a data processing method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining initial rendering data, wherein the initial rendering data is used for rendering a game scene picture to be displayed in a graphical user interface of a mobile terminal; dividing the initial rendering data into first part data and second part data based on the data type of the initial rendering data, wherein the first part data is compressible data in the initial rendering data, and the second part data is prohibited compression data in the initial rendering data; performing compression processing on the first part data to obtain compressed data; determining target rendering data by using the compressed data and the second part data; and performing a rendering operation based on the target rendering data. The application solves the technical problem of poor picture rendering performance.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a data processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] Currently, the common approach to addressing deferred rendering is to use a memoryless buffer. However, mobile devices have limitations on the amount of cache available, which leads to poor rendering performance due to the limited cache storage capacity.

[0003] There is currently no effective solution to the problem of poor image rendering performance. Summary of the Invention

[0004] At least some embodiments of the present invention provide a data processing method, apparatus, storage medium, and electronic device to at least solve the technical problem of poor image rendering performance.

[0005] According to one embodiment of the present invention, a data processing method is provided, comprising: acquiring initial rendering data, wherein the initial rendering data is used to render a game scene to be displayed within a graphical user interface of a mobile terminal; dividing the initial rendering data into a first part of data and a second part of data based on the data type of the initial rendering data, wherein the first part of data is compressible data in the initial rendering data and the second part of data is uncompressible data in the initial rendering data; compressing the first part of data to obtain compressed data; determining target rendering data using the compressed data and the second part of data; and performing a rendering operation based on the target rendering data.

[0006] Optionally, the first part of the data is compressed to obtain compressed data, including: dividing the first part of the data into a first type of data and a second type of data, wherein the first type of data is used to represent the corresponding normal data, and the second type of data is used to represent the data in the first part of the data other than the normal data; and compressing the first type of data and the second type of data to obtain compressed data.

[0007] Optionally, the first type of data is compressed to obtain compressed data, including: converting the first type of data into planar data to obtain first compressed data, wherein the number of channels in the first compressed data is less than the number of channels in the first type of data.

[0008] Optionally, the second type of data is compressed to obtain compressed data, including: determining the data to be saved in the second type of data based on the pixel position of the second type of data; and saving the data to be saved to obtain the second compressed data.

[0009] Optionally, determining the data to be saved in the second type of data based on the pixel position of the second type of data includes: determining the data content corresponding to the first position as the data to be saved in the second type of data in response to the pixel position being a first position; and determining the data content corresponding to the second position as the data to be saved in the second type of data in response to the pixel position being a second position.

[0010] Optionally, the target rendering data is determined using the compressed data and the second part of the data, including: performing a restoration process on the second compressed data to obtain the rendering data corresponding to the second compressed data; and obtaining the target rendering data based on the rendering data, the first compressed data, and the second part of the data.

[0011] Optionally, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data, including: determining the compressed data that is horizontally adjacent to the second compressed data and the compressed data that is vertically adjacent to the second compressed data based on an objective function, wherein the objective function is used to determine the difference between the pixel values ​​corresponding to the adjacent compressed data; and restoring the second compressed data based on the horizontally adjacent compressed data and the vertically adjacent compressed data to obtain the rendering data corresponding to the second compressed data.

[0012] Optionally, based on the horizontally adjacent compressed data and the vertically adjacent compressed data, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data. This includes: determining the difference between the brightness data in the second compressed data and the brightness data in the horizontally adjacent compressed data to obtain a first difference; determining the difference between the brightness data in the second compressed data and the brightness data in the vertically adjacent compressed data to obtain a second difference; and performing interpolation calculations on the horizontally adjacent compressed data and the vertically adjacent compressed data based on the first difference and the second difference to obtain the rendering data corresponding to the second compressed data.

[0013] Optionally, the difference between the pixel value of the second compressed data and the pixel value of the compressed data at adjacent positions satisfies a pixel threshold.

[0014] According to one embodiment of the present invention, a data processing apparatus is also provided, comprising: an acquisition unit for acquiring initial rendering data, wherein the initial rendering data is used to render a game scene to be displayed within a graphical user interface of a mobile terminal; a division unit for dividing the initial rendering data into a first part of data and a second part of data based on the data type of the initial rendering data, wherein the first part of data is compressible data in the initial rendering data and the second part of data is non-compressible data in the initial rendering data; a processing unit for compressing the first part of data to obtain compressed data; a determination unit for determining target rendering data using the compressed data and the second part of data; and an execution unit for performing a rendering operation based on the target rendering data.

[0015] According to one embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the data processing method described in any of the preceding claims when running.

[0016] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the data processing method described in any of the preceding claims.

[0017] In this embodiment of the invention, initial rendering data is obtained, which is used to render the game scene to be displayed within the graphical user interface of the mobile device. Based on the data type of the initial rendering data, it is divided into a first part of data and a second part of data, wherein the first part of data is compressible data and the second part of data is non-compressible data. The first part of data is compressed to obtain compressed data. The compressed data and the second part of data are used to determine the target rendering data. Rendering operations are performed based on the target rendering data. In other words, this embodiment of the invention classifies the initial rendering data according to its data type to obtain a first part of data and a second part of data, and compresses the first part of data, thereby compressing the initial rendering data, increasing the data storage capacity, and thus achieving the technical effect of improving the rendering performance of the screen, solving the technical problem of poor rendering performance. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a mobile terminal for a data processing method according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of a data processing method according to one embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a spherical projection method according to one embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of pixel-separated storage according to one embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of a 01 array of pixel positions according to one embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of the computation result of a central processing unit according to one embodiment of the present invention;

[0025] Figure 7 This is a schematic diagram illustrating the pixel-separated data storage according to one embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram showing the rendering completed according to one embodiment of the present invention;

[0027] Figure 9 This is a schematic diagram of the same material rendering according to one embodiment of the present invention;

[0028] Figure 10 This is a schematic diagram of different material rendering according to one embodiment of the present invention;

[0029] Figure 11 This is a structural block diagram of a data processing apparatus according to an embodiment of the present invention;

[0030] Figure 12 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0034] The rendering buffer can refer to a memory area on the central processing unit. Different types of data correspond to different memory areas, which can include the color buffer that stores material data and color data, the depth buffer that stores depth data, etc.

[0035] RenderTarget (RT): A texture map used to record the GPU rendering results. It is a block of memory on the GPU. For example, GBufferA, GBufferB, GBufferC, and GBufferD can be used to represent different render targets.

[0036] Deferred rendering is a rendering strategy that allows material data to be recorded on multiple rendering targets first, and then these material data are read uniformly to render the final lighting result.

[0037] Forward rendering is also a rendering strategy that can directly obtain all the data of a model and calculate all the lighting at the same time without saving a temporary set of rendering targets. Theoretically, this method has lower bandwidth, but the lighting effects are limited.

[0038] Bandwidth, the number of times depth memory and color memory are read and written during the rendering process (which can be the amount of data), increases power consumption and phone heat generation, and is an important indicator for performance evaluation.

[0039] Geometry Buffer (GBuffer for short): In deferred rendering, it is a group of rendering targets that temporarily store material data;

[0040] The memory (MemoryLess Buffer) on the central processing unit can be a cache type unique to the mobile phone. This type of cache can have very low bandwidth, but there is a limit to the amount.

[0041] Low-frequency image data can be data whose content does not change drastically with changes in position on the image;

[0042] High-frequency data refers to data whose content changes drastically with position on an image.

[0043] According to one embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] This method embodiment can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, the mobile terminal can be a smartphone (such as an Android phone, iOS phone, etc.), tablet computer, PDA, mobile Internet Device (MID), PAD, game console, and other terminal devices. Figure 1 This is a hardware structure block diagram of a mobile terminal for a data processing method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a central processing unit (CPU), graphics processing unit (GPU), digital signal processing (DSP) chip, microprocessor (MCU), programmable logic device (FPGA), neural network processor (NPU), tensor processor (TPU), artificial intelligence (AI) type processor, etc.) and a memory 104 for storing data are also shown. Optionally, the mobile terminal may further include a transmission device 106 for communication functions, an input / output device 108, and a display device 110. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned data processing method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0047] The inputs in input / output device 108 can come from multiple human interface devices (HIDs). Examples include keyboards and mice, gamepads, and other dedicated game controllers (such as steering wheels, fishing rods, dance mats, and remote controls). Some HIDs, in addition to providing input functions, can also provide output functions, such as force feedback and vibration from gamepads, and audio output from controllers.

[0048] Display device 110 may be, for example, a head-up display (HUD), a touchscreen liquid crystal display (LCD), and a touch display (also referred to as a "touchscreen" or "touch display"). The LCD allows a user to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which allows the user to interact with the GUI by touching and / or gesturing on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, a call interface, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0049] In one possible implementation, embodiments of the present invention provide a data processing method. Figure 2 This is a flowchart of a data processing method according to one embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0050] Step S202: Obtain initial rendering data, wherein the initial rendering data is used to render the game scene to be displayed in the graphical user interface of the mobile terminal.

[0051] In the technical solution provided by step S202 of the present invention, initial rendering data can be obtained from the storage location. For example, initial rendering data can be read from the geometry buffer (GBuffer) to achieve the purpose of obtaining initial rendering data. The initial rendering data can be used to render the game scene screen to be displayed in the graphical user interface of the mobile terminal, and can include material data, such as normal data, color data, custom data (CustomData) set according to different materials, material type data (MaterialID), roughness, shadow data (InShadow), depth data, and lighting data.

[0052] Optionally, rendering techniques can be divided into deferred rendering and forward rendering. Deferred rendering is the current trend in real-time rendering, which can achieve better lighting and material effects and is more flexible in production. It is now widely used in most console and PC games. Deferred rendering requires recording material data on multiple rendering targets. These rendering buffers that store data can be called geometry buffers (GBuffers). The initial rendering data can be obtained by reading the material data in the GBuffer.

[0053] For example, the necessary geometry buffer (GBuffer) needs to store 160 bits of initial rendering data, which may include: normal data (3*8 bits), color data (Diffuse) (3*8 bits), custom data (CustomData) set according to different materials (3*8 bits, such as metal data (Metallic) stored in the x channel, thickness data (Thickness) stored in the y channel, material type data (MaterialID) (8 bits), roughness data (Roughness) (8 bits), shadow data (InShadow) (8 bits), depth data (Depth) (16 bits), and lighting data (Lighting) (3*16 bits).

[0054] It should be noted that the initial rendering data mentioned above is only for illustrative purposes and no specific limitations are imposed here.

[0055] Step S204: Based on the data type of the initial rendering data, the initial rendering data is divided into a first part of data and a second part of data, wherein the first part of data is the compressible data in the initial rendering data and the second part of data is the non-compressible data in the initial rendering data.

[0056] In the technical solution provided by step S204 of the present invention, the data type of the obtained initial rendering data is determined, and the initial rendering data is divided based on the data type to obtain a first part of data and a second part of data. The data type can be compressible data and incompressible data (incompressible data). The first part of data can be compressible data, such as low-frequency data such as color data (Diffuse). The second part of data is the compressible data in the initial rendering data, and can be incompressible data that needs to be accurately saved, such as lighting data, depth data, and shadow data (InShadow).

[0057] Optionally, the data type of the initial rendering data is determined. When the initial rendering data is compressible, it is divided into a first part of the data; when the initial rendering data is incompressible, it is divided into a second part of the data. For example, when the initial rendering data is low-frequency data, it can be divided into a first part of the data; when the initial rendering data is depth data, it can be determined as a second part of the data.

[0058] In this embodiment, since lighting data, depth data, and shadow data are data that need to be accurately saved, these three types of data cannot be compressed. At the same time, material type data is marker data and cannot be compressed either. Therefore, in this embodiment of the invention, normal data, color data, custom data, and roughness data can be compressed.

[0059] Optionally, embodiments of the present invention divide the initial rendering data using data types to obtain compressible data and incompressible data. By compressing the compressible data, the amount of data is reduced, thereby improving the rendering effect of the game.

[0060] Step S206: Compress the first part of the data to obtain compressed data.

[0061] In the technical solution provided by step S206 of the present invention, the compressible first part of the data is compressed to obtain compressed data. The compression of the first part of the data can be used to reduce the amount of data, for example, by using the characteristics of the data itself to perform channel merging, thereby achieving the purpose of data compression; it can also be used to compress the data by utilizing the characteristics of image storage, etc.

[0062] Currently, to enable deferred rendering on mobile phones and address bandwidth and power consumption issues, mobile phones typically use memoryless buffers to render game scenes. Game engines limit the number of GBuffers. However, due to the strict limitations on GBuffer counts on mobile phones, game engine design needs to be backward compatible to support deferred rendering on most devices. For example, the pixel data in each memoryless buffer on a mobile phone cannot exceed 128 bits, or some phones only support four render targets. For some mobile phone CPU architectures, the single-pixel storage of four GBuffers exceeds 128 bits, making it impossible to fully utilize the memoryless buffer feature, resulting in poor rendering performance on some phones. To address these issues, in this embodiment of the invention, the initial rendering data is classified to obtain a compressible first part of the data. By compressing this first part of the data, the initial rendering data is compressed.

[0063] Step S208: Determine the target rendering data using the compressed data and the second part of the data.

[0064] In the technical solution provided by step S208 of the present invention, compressed data and second part data (uncompressed data) are determined to obtain target rendering data, wherein the target rendering data is used to render the game scene screen to be displayed in the graphical user interface of the mobile terminal.

[0065] Optionally, the target rendering data includes compressed data and a second part of the data; therefore, the target rendering data can be determined using the compressed data and the second part of the data.

[0066] Step S210: Perform rendering operations based on the target rendering data.

[0067] In the technical solution provided by step S210 of the present invention, the target rendering data is used to render the game scene screen to be displayed in the graphical user interface of the mobile terminal. The target rendering data performs a rendering operation to obtain the game scene screen.

[0068] Through steps S202 to S210, initial rendering data is obtained, which is used to render the game scene to be displayed within the graphical user interface of the mobile terminal. Based on the data type of the initial rendering data, it is divided into a first part of data and a second part of data. The first part of data consists of compressible data, and the second part consists of uncompressible data. The first part of data is compressed to obtain compressed data. The compressed data and the second part of data are used to determine the target rendering data. Rendering operations are then performed based on the target rendering data. In other words, this invention classifies the initial rendering data according to its data type to obtain a first part of data and a second part of data. By compressing the first part of data, the initial rendering data is compressed, increasing the data storage capacity and thus improving the technical effect of poor rendering performance, solving the technical problem of poor rendering performance.

[0069] The embodiments of the present invention will be further described below.

[0070] As an optional embodiment, step S206, compressing the first part of the data to obtain compressed data, includes: dividing the first part of the data into a first type of data and a second type of data, wherein the first type of data is used to represent the corresponding normal data, and the second type of data is used to represent the data in the first part of the data other than the normal data; and compressing the first type of data and the second type of data to obtain compressed data.

[0071] In this embodiment, since normal data can be compressed using image storage characteristics, while other data cannot be compressed using image storage characteristics, the first part of the data can be divided into a first type of data and a second type of data. The first type of data can be used to characterize the corresponding normal data, and the second type of data can be used to characterize the data in the first part of the data other than normal data, such as color data in low-frequency data.

[0072] Optionally, the first part of the data may include data that can be channel-merged using the characteristics of the data itself and data that can be compressed using the characteristics of image storage. The first part of the data is divided into a first type of data that can be compressed using the characteristics of image storage and data that can be channel-merged using the characteristics of the data itself.

[0073] Optionally, since low-frequency data does not vary significantly within a certain range, data channels can be merged according to the characteristics of the data itself. For example, different data can be stored in one channel or using adjacent pixels according to the characteristics of the material data. On the other hand, for normals, which are unit vectors, they can be compressed and encoded through two channels. Therefore, the first part of the data is divided into first type data and second type data, and different compression methods are selected for different types of data, thereby achieving the goal of improving data compression efficiency.

[0074] As an optional embodiment, step S206, compressing the first type of data to obtain compressed data, includes: converting the first type of data into planar data to obtain first compressed data, wherein the number of channels in the first compressed data is less than the number of channels in the first type of data.

[0075] In this embodiment, the first type of data can be normal data, which generally requires three channels for storage. Since the normal data itself is normalized data, the third channel can be omitted from the perspective of data storage. Therefore, the first type of data is converted into planar data with a maximum of two channels to obtain the first compressed data.

[0076] In this embodiment of the invention, by converting the first type of data from three-dimensional space data to planar data, the purpose of omitting the third channel and compressing the data is achieved. Therefore, the number of channels of the obtained first compressed data must be less than the number of channels of the first type of data; otherwise, the purpose of data compression cannot be achieved.

[0077] Alternatively, to preserve more accuracy, a spherical projection strategy can be used to process the normal data to obtain planar information of the normal data, which can yield data for up to two channels.

[0078] For example, the normal data can be set to any point P on the sphere. Connect point P and reference point N. The projection point P' on the plane circle is the converted data. At this time, the projection point only has planar information, so at most two channels are needed, thus achieving the purpose of data compression.

[0079] As an optional embodiment, step S206, compressing the second type of data to obtain compressed data, includes: determining the data to be saved in the second type of data based on the pixel position of the second type of data; saving the data to be saved to obtain the second compressed data.

[0080] In this embodiment, the pixel position of the second type of data is determined. Based on the pixel position of the second type of data, the data to be saved in the second type of data is determined. The data to be saved is saved to obtain the second compressed data. The data to be saved can be data that is set in advance according to operating habits or needs. For example, when the pixel position is at the flag bit 0, the corresponding data to be saved can be GBufferC1; when the pixel position is at the flag bit 1, the corresponding data to be saved can be GBufferC2. The pixel position can be represented by texPos.

[0081] Optionally, for the same material, the low-frequency data similar to the color data in the second type of data may also include: metallic, roughness data, and custom data with low utilization. The low-frequency data can be saved together with the color data.

[0082] Optionally, different data to be saved are set for different pixel positions. The pixel position (texPos) of the second type of data is determined by using coordinates to determine the pixel position of the data, thereby determining the data to be saved in the second type of data. The data to be saved is then saved to obtain the second compressed data.

[0083] As an optional embodiment, determining the data to be saved in the second type of data based on the pixel position of the second type of data includes: in response to the pixel position being a first position, determining the data content corresponding to the first position as the data to be saved in the second type of data; and in response to the pixel position being a second position, determining the data content corresponding to the second position as the data to be saved in the second type of data.

[0084] In this embodiment, the pixel position of the second type of data is determined. In response to the pixel position being the first position, the data content to be saved set at the first position is determined as the data to be saved in the second type of data. In response to the pixel position being the second position, the data content to be saved set at the second position is determined as the data to be saved in the second type of data. The first position can be a coordinate position with a corresponding coordinate position identifier of odd or even number, and the second position can be a coordinate position with a corresponding coordinate position identifier of even or odd number. The first position and the second position are adjacent.

[0085] Optionally, different regions in texture coordinates (uv coordinates) can be represented by two-dimensional positive integer data. For example, different regions can be represented by 0 and 1, which can correspond to GbufferC1 and GbufferC2 respectively. The data in GbufferC1 and GbufferC2 are stored alternately. The pixel position of the currently processed data can be determined by the numbers corresponding to the data regions. It should be noted that the above numbers are only illustrative examples and are not specifically limited here.

[0086] For example, the input pixel position is used to determine whether the coordinates of the second type of data pixel position correspond to an odd or even number, or 0 or 1, based on the modulo operation. Based on the odd or even number or 0 or 1 of this two-dimensional coordinate, the data content to be saved for the second type of data is determined. If the two-dimensional coordinate of the pixel position is 0, then the pixel is saved in GBufferC1. If the two-dimensional coordinate of the pixel position is 1, then the pixel is saved in GBufferC2.

[0087] Optionally, since the brightness data in the second type of data is high-frequency data, the full amount of brightness data can be retained; since color changes are low-frequency data, the precision of color data storage can be reduced. In this embodiment of the invention, spatial domain compression is performed by storing the texture space at pixel intervals, thereby achieving the purpose of data compression.

[0088] In related technologies, color data compression usually involves converting the data to color spaces such as Hue Saturation Value (HSV), color mode (Lab), Hue Saturation Lightness (HSL), and YCbCr.

[0089] In this embodiment of the invention, the color data of the three primary colors (Red Green Blue, abbreviated as RGB) can be converted into the color space (YCbCr) and encoded. After encoding, the brightness data stored in the Y channel is saved to the x channel of the buffer area GBuffer C, and the blue chromaticity component (Cb) channel and the red chromaticity component (Cr) channel of the color data are compressed together with the subsequent data using pixel-skipping compression.

[0090] Optionally, two sets of pixel-skipping compressed data can be configured to be alternately stored in GBufferC, for example, GBufferC1 and GBufferC2. GBufferC1 can be configured to store luminance data (Diffuse.Y), the blue chromaticity component of the Diffuse (Diffuse.Cb), roughness data, and custom data (CustomData.y), which can be:

[0091] GBufferC1: Diffuse.Y, Diffuse.Cb, Roughness, CustomData.y

[0092] You can configure GBufferC2 to store luminance data (Diffuse.Y), red chromaticity component (Diffuse.Cr), metal data from custom data (CustomData.x(Metallic)), and custom data (CustomData.z). The options are:

[0093] GBufferC2: Diffuse.Y, Diffuse.Cr, CustomData.x(Metallic), CustomData.z

[0094] In this embodiment, the data content to be saved at the first position and the data content to be saved at the second position are saved alternately. By determining the pixel position of the data, the data content is selectively saved, thereby achieving the purpose of data compression.

[0095] It should be noted that the data content to be saved in GBufferC1 and GBufferC2 mentioned above is only for illustrative purposes and no specific restrictions are imposed here.

[0096] As an optional embodiment, step S208, determining the target rendering data using compressed data and the second part of the data, includes: performing restoration processing on the second compressed data to obtain rendering data corresponding to the second compressed data; and obtaining the target rendering data based on the rendering data, the first compressed data, and the second part of the data.

[0097] In this embodiment, since mobile phones typically use a memoryless buffer to store the second compressed data, the data stored in the memoryless buffer can only exist on the memory of the on-chip for local use and cannot be read directly. Therefore, it is necessary to restore the second compressed data to obtain the rendering data corresponding to the second compressed data. Based on the rendering data, the first compressed data, and the second part of the data, the target rendering data can be obtained.

[0098] In this embodiment, since the data stored in GBufferC1 and GBufferC2 are stored alternately, the data around GBufferC1 is stored in GBufferC2. Therefore, the operation of restoring the second compressed data is to restore the missing cached data from the surrounding geometric buffer area.

[0099] As an optional embodiment, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data, including: determining the compressed data that is horizontally adjacent to the second compressed data and the compressed data that is vertically adjacent to the second compressed data based on an objective function, wherein the objective function is used to determine the difference between the pixel values ​​corresponding to the adjacent compressed data; and restoring the second compressed data based on the horizontally adjacent compressed data and the vertically adjacent compressed data to obtain the rendering data corresponding to the second compressed data.

[0100] In this embodiment, mobile phones typically use a memory-free buffer to store the second compressed data. However, the data stored in the memory-free buffer can only exist on the memory of a single-chip buffer for local use and cannot be directly read. There is also the problem that adjacent stored data cannot be directly determined. However, in this embodiment of the invention, adjacent stored data is needed to restore the second compressed data. Therefore, an objective function can be used to determine the compressed data that is horizontally adjacent to the second compressed data and the compressed data that is vertically adjacent to the second compressed data. Based on the horizontally adjacent compressed data and the vertically adjacent compressed data, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data. The objective function can be used to determine the difference between the pixel values ​​corresponding to adjacent compressed data, and can be a partial derivative function in the hardware computing characteristics of the central processing unit.

[0101] Optionally, the CPU's operations are submitted and run in groups of threads. At the pixel level, the execution of threads is based on a quad (2×2 pixels) as the smallest unit of organization. The CPU can calculate the derivative by taking the difference between pixel values ​​in the quad. For example, dFdx(p(x,y)) = p(x+1,y) - p(x,y) or dFdx(p(x,y)) = p(x,y+1) - p(x,y). Here, p(x,y) can be the pixel value of the second compressed data, then p(x+1,y) can be the pixel value of the data to the right of p(x,y), p(x,y+1) can be the pixel value of the data below p(x,y), and so on.

[0102] Optionally, by using the partial derivatives provided by the central processing unit to calculate the second compressed data, the difference between the second compressed data and the results of adjacent pixels can be obtained, thereby determining the GBufferC1 or GBufferC2 corresponding to the adjacent pixels.

[0103] For example, if InGBufferC is set as the second compressed data, and the second compressed data is the top-left corner data, then Neighbor_H can be the horizontally adjacent pixel of the second compressed data, which can be the left or right pixel. Neighbor_V can be the vertically adjacent pixel of the second compressed data, which can be the bottom or top pixel. Whether it is left, right, top, or bottom depends on the position of the second compressed data; there are no specific restrictions here. Neighbor_H can be calculated by adding the partial derivative of the second compressed data, which can be calculated using the following formula:

[0104] Neighbor_H=InGBufferC+ddx(InGBufferC)

[0105] Neighbor_V can be the partial derivative of the second compressed data plus the second compressed data, and can be calculated using the following formula:

[0106] Neighbor_V=InGBufferC+ddy(InGBufferC)

[0107] For example, when the second compressed data is the data in the top right corner, Neighbor_H can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, which can be calculated using the following formula:

[0108] Neighbor_H=InGBufferC-ddx(InGBufferC)

[0109] Neighbor_V can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, using the following formula:

[0110] Neighbor_V=InGBufferC-ddy(InGBufferC)

[0111] For example, when the second compressed data is the bottom left corner data, Neighbor_H can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, which can be calculated using the following formula:

[0112] Neighbor_H=InGBufferC-ddx(InGBufferC)

[0113] Neighbor_V can be the second compressed data plus its partial derivative, and can be calculated using the following formula:

[0114] Neighbor_V=InGBufferC+ddy(InGBufferC)

[0115] For example, when the second compressed data is the data in the bottom right corner, Neighbor_H can be calculated by adding the partial derivative of the second compressed data to the second compressed data, which can be calculated using the following formula:

[0116] Neighbor_H=InGBufferC+ddx(InGBufferC)

[0117] Neighbor_V can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, using the following formula:

[0118] Neighbor_V=InGBufferC-ddy(InGBufferC)

[0119] Optionally, in the embodiments of the present invention, if the pixel attributes of adjacent positions are similar and the relative data store the same attribute, then the same attribute calculated by p(x,y) and p(x,y+1) is similar, and the difference between different attributes is similar to p(x+1,y)-p(x,y)-p(x+1,y+1)-p(x+1,y), thereby it can be deduced that p(x+1,y+1) is approximately p(x,y+1)+p(x+1,y)-p(x,y).

[0120] It should be noted that the above addition and subtraction operations depend on the different calculation strategies of the partial derivative function (ddx). No specific restrictions are placed on the calculation of ddx here, and the calculation method of adjacent pixels can be changed according to the calculation method of ddx.

[0121] As an optional embodiment, based on horizontally adjacent compressed data and vertically adjacent compressed data, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data. This includes: determining the difference between the brightness data in the second compressed data and the brightness data in the horizontally adjacent compressed data to obtain a first difference; determining the difference between the brightness data in the second compressed data and the brightness data in the vertically adjacent compressed data to obtain a second difference; and performing interpolation calculations on the horizontally adjacent compressed data and the vertically adjacent compressed data based on the first difference and the second difference to obtain the rendering data corresponding to the second compressed data.

[0122] In this embodiment, an objective function is used to determine the compressed data that is horizontally adjacent to the second compressed data and the compressed data that is vertically adjacent to the second compressed data, to determine the difference between the brightness data in the second compressed data and the brightness data in the horizontally adjacent compressed data, to obtain a first difference, and to determine the difference between the brightness data in the second compressed data and the brightness data in the vertically adjacent compressed data, to obtain a second difference. The first difference can be represented by biasL, and the second difference can be represented by biasU.

[0123] Optionally, in this embodiment of the invention, the storage can be in the form of a checkerboard grid, so the GBufferC1 position only stores the data content that needs to be saved in GBufferC1, and the data content that needs to be saved in GBufferC2 is missing. The missing data content in GBufferC1 can be obtained by linear interpolation of the calculated adjacent compressed data.

[0124] Optionally, the difference between the luminance data (Current.x) in the second compressed data and the luminance data (Left.x) in the horizontally adjacent compressed data is calculated to obtain a first difference (biasL). The difference between the luminance data (Current.x) in the second compressed data and the luminance data (Up.x) in the vertically adjacent compressed data is calculated to obtain a second difference (biasU). The missing data content in the second compressed data is obtained by adding the result of dividing the vertical data by the second difference and the result of dividing the horizontal data by the first difference, and then dividing the sum by the sum of the first and second differences. This can be calculated using the following formula:

[0125] (Up*biasU+Left*biasL) / (biasU+biasL)

[0126] To avoid division by zero, the absolute values ​​of the first and second differences need to be taken. When either the first or second difference is 0, it can be taken as 0.001, which can be:

[0127] biasU=max(0.01, abs(Current.x-Up.x))

[0128] biasL=max(0.01, abs(Current.x-Left.x))

[0129] In this context, Current can be used to represent the second compressed data, Current.x represents the luminance data, and other components (such as y, z, w) can represent GBufferC1 or GBufferC2; Up and Left can represent the left and top data respectively, the x component represents the luminance data, and other components (such as y, z, w) can represent the missing GBufferC1 or GBufferC2 in the second compressed data.

[0130] Optionally, by using the first difference and the second difference, interpolation calculations are performed on the horizontally adjacent compressed data and the vertically adjacent compressed data to restore the second compressed data and achieve the purpose of obtaining the complete rendering data corresponding to the second compressed data.

[0131] As an optional embodiment, the difference between the pixel value of the second compressed data and the pixel value of the compressed data at adjacent positions satisfies a pixel threshold.

[0132] In this embodiment of the invention, the difference between the pixel value of the second compressed data and the pixel value of the compressed data at the adjacent position satisfies a pixel threshold, wherein the pixel threshold can be a value set according to actual needs.

[0133] Optionally, saving different materials at pixel intervals can lead to material loss. However, if the difference between the pixel value of the second compressed data and the pixel value of the compressed data at adjacent positions meets the pixel threshold, material loss can be avoided.

[0134] It should be noted that since color components are low-frequency data, their changes are not obvious in the image. Storing them at a lower resolution can save a lot of space. However, if the resolution of the geometry buffer (GBuffer) is different, uncontrollable abnormal effects may appear at the boundaries, such as banding color layering and aliasing. Therefore, in this embodiment of the invention, the original resolution can be used for storage. Optionally, when sudden high-frequency changes occur at the boundaries of different materials, anti-aliasing can be used to solve the problem of abnormal image rendering caused by sudden high-frequency changes at the boundaries of different materials.

[0135] In this embodiment of the invention, the initial rendering data is classified according to the data type to obtain a first part of data and a second part of data. The first part of data is compressed, thereby compressing the initial rendering data, increasing the data storage capacity, and thus achieving the technical effect of improving the rendering performance of the screen, solving the technical problem of poor rendering performance.

[0136] The technical solutions of the present invention will be further illustrated below with reference to preferred embodiments. Specifically, a design scheme for a satellite buffer (MicroGbuffer) format for mobile phone deferred rendering will be described.

[0137] Currently, in game rendering, rendering technology can be divided into deferred rendering and forward rendering. Deferred rendering is the current trend in real-time rendering, which can achieve better lighting and material effects and is more flexible in production. It is now widely used in most console and PC games. Although mobile games are also gradually transitioning to deferred rendering, some problems still exist.

[0138] Deferred rendering requires recording material data on multiple rendering targets. These rendering buffers that store the data can be called geometry buffers (GBuffers). By uniformly reading the material data in the GBuffers and rendering, the final lighting result can be obtained. However, on mobile phones, deferred rendering has a large bandwidth and power consumption, which can cause heat generation and frequency reduction, limiting the use of deferred rendering on mobile phones and also limiting the rendering effect of games.

[0139] To enable deferred rendering on mobile devices and address bandwidth and power consumption issues, mobile phones typically use memoryless buffers to render game scenes. Game engines limit the number of GBuffers. However, due to the strict limitations on GBuffer counts on mobile devices, game engines need to be backward compatible to support deferred rendering on most phones. For example, some phones cannot store more than 128 bits of pixel data per memoryless buffer, and some only support four render targets. Therefore, memoryless technology on mobile phones presents the following problems: For some mobile CPU architectures, the storage of a single pixel in four GBuffers exceeds 128 bits, making it impossible to fully utilize the memoryless GBuffer functionality, resulting in poor rendering performance on some phones; The four-channel GBuffer contains a lot of low-frequency data, which is wasteful to store general material data, reducing GBuffer utilization and leading to a large amount of rendering data.

[0140] In the methods described above, the 4RT GBuffer of a game engine (Unity) is already the limit. For 3D modeling and rendering game engines (such as UE4), the deferred pipeline needs to be reconstructed on mobile devices. To increase the amount of rendering data stored in this embodiment, on the one hand, since low-frequency data does not change significantly within a certain range, pixel-by-pixel or single-pixel compression can achieve good results. Therefore, data channels are merged according to the characteristics of the data itself. For example, different data can be stored in one channel or using adjacent pixels based on the characteristics of the material data. On the other hand, image storage characteristics are utilized to compress a certain data type. For example, for a normal, which is a unit vector, it can be compressed and encoded using only two channels. Through these two aspects, this embodiment achieves the goal of further compressing the GBuffer, for example, compressing it to store data using only two texture buffers (Multi-Render Target 2, or MRT2 for short) or 3. Simultaneously, the single-pixel storage space is compressed to within 128 bits, thereby reducing the amount of data in the rendering target and improving the rendering effect of the game.

[0141] The embodiments of the present invention will be further described below.

[0142] In this embodiment, the necessary GBuffer needs to store a total of 160 bits of content, which may include: Normal (3*8 bits), Diffuse (3*8 bits), Custom Data (3*8 bits) set according to different materials, such as Metallic data stored in the x channel, Thickness data stored in the y channel, Material ID (8 bits), Roughness (8 bits), InShadow (8 bits), Depth (16 bits), and Lighting (3*16 bits).

[0143] In this embodiment, since lighting data, depth data, and shadow data are data that need to be accurately saved, these three types of data cannot be compressed. At the same time, material type data is marker data and cannot be compressed either. Therefore, in this embodiment of the invention, normal data, color data, custom data, and roughness data can be compressed.

[0144] As an optional implementation, the normal data is compressed.

[0145] In related technologies, the normal data size is 3*8 bits, and it generally needs to occupy three channels for storage. Since Normal data itself is normalized, the third channel can be omitted from the perspective of data storage.

[0146] Alternatively, to preserve more accuracy, a spherical projection strategy can be used to process the normal data to obtain planar information of the normal data, thus yielding data for up to two channels. Figure 3 This is a schematic diagram of a spherical projection method according to one embodiment of the present invention, as shown below. Figure 3 As shown, the normal data can be set to any point P on the sphere. Connecting point P and the reference point N, the projection point P' on the plane circle is the converted data. Figure 3 As shown, the projection points only have planar information, so only two channels are needed, thus achieving the purpose of data compression.

[0147] As an optional implementation, the second type of data is compressed.

[0148] In this embodiment, compressing the second type of data may include compressing low-frequency data such as color data.

[0149] Optionally, since brightness is high-frequency data, the full brightness data can be retained; since color change is low-frequency data, the precision of color data storage can be reduced. In this embodiment of the invention, spatial domain compression is performed by storing the texture space at pixel intervals, thereby achieving the purpose of data compression.

[0150] In related technologies, color data compression usually involves converting the data to color spaces such as Hue Saturation Value (HSV), color mode (Lab), Hue Saturation Lightness (HSL), and YCbCr.

[0151] It should be noted that since color components are low-frequency data, their changes are not obvious in the image. Storing them at a lower resolution can save a lot of space. However, if the resolution of the geometry buffer (GBuffer) is different, uncontrollable abnormal effects may appear at the boundaries, such as banding color layering and aliasing. Therefore, in this embodiment of the invention, the original resolution can be used for storage. Figure 4 This is a schematic diagram of pixel-separated storage according to one embodiment of the present invention, as shown below. Figure 4As shown, positions with the same color store the same data content, positions with different colors store different data content, and positions separated by pixels store different data content.

[0152] Optionally, when sudden high-frequency changes occur at different material boundaries, anti-aliasing can be used to solve the problem of sudden high-frequency changes at different material boundaries.

[0153] Optionally, for the same material, low-frequency data similar to color data may also include: metallic, roughness, and custom data with low utilization. The above low-frequency data can be saved together with the diffuse.

[0154] Optionally, the three primary colors (Red, Green, Blue, or RGB) of the Diffuse are converted to the color space (YCbCr) and encoded. After encoding, the luminance data stored in the Y channel is saved to the x channel of the buffer area GBuffer C, and the blue chromaticity component (Cb) channel and the red chromaticity component (Cr) channel of the Diffuse are compressed together with the subsequent data using pixel-skipping compression.

[0155] Optionally, two sets of pixel-skipping compressed data can be configured to be alternately stored in GBufferC. For example, GBufferC 1 and GBufferC 2 can be configured. GBufferC1 can be configured to store luminance data (Diffuse.Y), blue chromaticity component of Diffuse (Diffuse.Cb), roughness, and custom data (CustomData.y).

[0156] Optionally, you can configure GBufferC2 to store luminance data (Diffuse.Y), red chromaticity component (Diffuse.Cr), metal data (CustomData.x(Metallic)) from custom data, and custom data (CustomData.z).

[0157] In this embodiment, since the data stored in GBufferC1 and GBufferC2 are stored alternately, the pixels surrounding GBufferC1 store GBufferC2. Therefore, the operation of restoring compressed data is to restore the missing cached data from the surrounding geometric buffer area (GBufferC).

[0158] Optionally, the pixel position (texPos) of the second type of data can be determined using coordinates. Different regions in the uv coordinate system can be represented by two-dimensional positive integer data, for example... Figure 5 This is a schematic diagram of a 01 array of pixel positions according to one embodiment of the present invention, as shown below. Figure 5 As shown, different regions can be represented by 0 and 1. 0 and 1 can correspond to GbufferC1 and GbufferC2 respectively. Data in GbufferC1 and GbufferC2 are stored alternately. The pixel position of the currently processed data can be determined by the number corresponding to the data region. It should be noted that the above numbers are only for illustrative purposes and are not specifically limited here.

[0159] For example, the input pixel position is used to determine whether the coordinates of the pixel position (texPos) of the second type of data are odd or even based on the modulo operation. Finally, based on whether the two-dimensional coordinates are odd or even, the data content to be saved for the second type of data is determined. If the two-dimensional coordinates of the pixel position are 0, the pixel is saved in GBufferC1. If the two-dimensional coordinates of the pixel position are 1, the pixel is saved in GBufferC2.

[0160] As an optional embodiment, the compressed second type of data is restored.

[0161] Since mobile phones typically use a memoryless buffer to store compressed second-type data, the data stored in the memoryless buffer can only exist on the on-chip memory for local use and cannot be directly read. This presents a problem of not being able to directly determine adjacent stored data. However, in this embodiment of the invention, adjacent stored data is needed to restore the compressed second-type data. Therefore, the partial derivative function in the hardware computing characteristics of the central processing unit can be used for restoration.

[0162] Optionally, the CPU's operations are submitted and executed in groups of threads. At the pixel level, the execution of a thread is organized using a quad (2×2 pixels) as the smallest unit. Figure 6 This is a schematic diagram of the computation result of a central processing unit according to one embodiment of the present invention, such as... Figure 6As shown, the central processing unit calculates the derivative by taking the difference between pixel values ​​in the packaged device. For example, dFdx(p(x,y))=p(x+1,y)-p(x,y) or dFdx(p(x,y))=p(x,y+1)-p(x,y), where p(x,y) can be the pixel value of the current data, then p(x+1,y) can be the pixel value of the data to the right of p(x,y), p(x,y+1) can be the pixel value of the data below p(x,y), and so on.

[0163] Optionally, by using the partial derivatives provided by the central processing unit to calculate the second compressed data GBufferC, the difference between the second compressed data and the results of adjacent pixels can be obtained, thereby determining the GBufferC1 or GBufferC2 corresponding to the adjacent pixels.

[0164] For example, such as Figure 8 As shown, InGBufferC is set to the top-left corner of the second compressed data. Neighbor_H can be the horizontally adjacent pixel of the second compressed data, which can be the left or right pixel. Neighbor_V can be the vertically adjacent pixel of the second compressed data, which can be the bottom or top pixel. Whether it is left, right, top, or bottom depends on the position of the second compressed data; there are no specific restrictions here. Neighbor_H can be calculated by adding the partial derivative of the second compressed data, which can be:

[0165] Neighbor_H=InGBufferC+ddx(InGBufferC)

[0166] Neighbor_V can be the partial derivative of the second compressed data plus the second compressed data, and can be:

[0167] Neighbor_V=InGBufferC+ddy(InGBufferC)

[0168] For example, such as Figure 8 As shown, when the second compressed data is the data in the upper right corner, Neighbor_H can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, which can be:

[0169] Neighbor_H=InGBufferC-ddx(InGBufferC)

[0170] Neighbor_V can be the partial derivative of the second compressed data minus the second compressed data, and can be:

[0171] Neighbor_V=InGBufferC-ddy(InGBufferC)

[0172] For example, such as Figure 8 As shown, when the second compressed data is the data in the lower left corner, Neighbor_H can be calculated by subtracting the partial derivative of the second compressed data from the second compressed data, which can be:

[0173] Neighbor_H=InGBufferC-ddx(InGBufferC)

[0174] Neighbor_V can be the second compressed data plus its partial derivative, and can be:

[0175] Neighbor_V=InGBufferC+ddy(InGBufferC)

[0176] For example, such as Figure 8 As shown, when the second compressed data is the data in the lower right corner, Neighbor_H can be calculated by adding the partial derivative of the second compressed data to the second compressed data, which can be:

[0177] Neighbor_H=InGBufferC+ddx(InGBufferC)

[0178] Neighbor_V can be the partial derivative of the second compressed data minus the second compressed data, and can be:

[0179] Neighbor_V=InGBufferC-ddy(InGBufferC)

[0180] Optionally, in the embodiments of the present invention, if the pixel attributes of adjacent positions are similar and the relative data store the same attribute, then the same attribute calculated by p(x,y) and p(x,y+1) is similar, and the difference between different attributes is similar to p(x+1,y)-p(x,y)-p(x+1,y+1)-p(x+1,y), thereby it can be deduced that p(x+1,y+1) is approximately p(x,y+1)+p(x+1,y)-p(x,y).

[0181] It should be noted that the above addition and subtraction operations depend on the different calculation strategies of the partial derivative function (ddx). No specific restrictions are placed on the calculation of ddx here, and the calculation method of adjacent pixels can be changed according to the calculation method of ddx.

[0182] In an embodiment of the present invention, Figure 7 This is a schematic diagram illustrating the pixel-separated data storage according to one embodiment of the present invention, such as... Figure 7As shown, compressed data can be stored pixel-by-pixel in a checkerboard pattern, so GBufferC1 only stores the data content that needs to be saved in GBufferC1 (corresponding to...). Figure 7 The data that needs to be saved in GBufferC2 is missing (corresponding to C in GBufferC2). Figure 7 The missing data in GBufferC1 can be obtained by linear interpolation of the calculated data above and the data on the right.

[0183] Optionally, the difference between the luminance data (Current.x) in the second compressed data and the luminance data (Left.x) in the horizontally adjacent compressed data is calculated to obtain a first difference (biasL). The difference between the luminance data (Current.x) in the second compressed data and the luminance data (Up.x) in the vertically adjacent compressed data is calculated to obtain a second difference (biasU). The missing data content in the second compressed data is obtained by adding the result of dividing the vertical data by the second difference and the result of dividing the horizontal data by the first difference, and then dividing the sum by the sum of the first and second differences.

[0184] (Up*biasU+Left*biasL) / (biasU+biasL)

[0185] To avoid division by zero, the absolute values ​​of the first and second differences need to be taken. When either the first or second difference is 0, it can be taken as 0.001, which can be:

[0186] biasU=max(0.01, abs(Current.x-Up.x))

[0187] biasL=max(0.01, abs(Current.x-Left.x))

[0188] In this context, Current can be used to represent the second compressed data, Current.x represents the luminance data, and other components (such as y, z, w) can represent GBufferC1 or GBufferC2; Up and Left can represent the left and top data respectively, the x component represents the luminance data, and other components (such as y, z, w) can represent the missing GBufferC1 or GBufferC2 in the second compressed data.

[0189] Optionally, by performing interpolation calculations on the horizontally adjacent compressed data and the vertically adjacent compressed data in the above manner, the restoration of the second compressed data is completed, thereby achieving the goal of obtaining the complete rendering data corresponding to the second compressed data.

[0190] Figure 8 This is a schematic diagram showing the completed rendering according to one embodiment of the present invention, as shown below. Figure 8 As shown, the graphical user interface is rendered based on complete Type II data, normal data, and uncompressed data, and the rendering result is obtained.

[0191] Figure 9 This is a schematic diagram of the same material rendering according to one embodiment of the present invention, such as... Figure 9 As shown, the rendered result has no obvious errors. 10 is a schematic diagram of rendering different materials according to one embodiment of the present invention, as shown. Figure 10 As shown, different types of materials exhibit material loss issues.

[0192] In this embodiment of the invention, the GBuffer content is compressed from 160 bits to 128 bits, a compression of 20%, while ensuring that the entire phone uses MemoryLessGBuffer. In this embodiment of the invention, since the calculation of the partial derivative function is used, all data already exists in the corresponding packaging device. If sampling of adjacent pixels is considered, the total number of position samplings remains unchanged. Since the use of inter-pixel buffer (texture buffer) reduces the number of GBuffers, thus saving more total bandwidth.

[0193] In this embodiment, data channels are merged according to the characteristics of the data itself. Based on the characteristics of the material data, different data are stored in one channel or using adjacent pixels. At the same time, image storage characteristics are used to compress a certain data type, thereby achieving the technical effect of improving the rendering effect and solving the technical problem of poor rendering performance.

[0194] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0195] This embodiment also provides a data processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "unit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0196] Figure 11 This is a structural block diagram of a data processing apparatus according to an embodiment of the present invention, such as... Figure 11 As shown, the data processing device may include: an acquisition unit 1102, a division unit 1104, a processing unit 1106, a determination unit 1108, and an execution unit 1110.

[0197] The acquisition unit 1102 is used to acquire initial rendering data, wherein the initial rendering data is used to render the game scene screen to be displayed in the graphical user interface of the mobile terminal.

[0198] The partitioning unit 1104 is used to divide the initial rendering data into a first part of data and a second part of data based on the data type of the initial rendering data. The first part of data is compressible data in the initial rendering data, and the second part of data is non-compressible data in the initial rendering data.

[0199] Processing unit 1106 is used to compress the first part of the data to obtain compressed data;

[0200] The determining unit 1108 is used to determine the target rendering data using the compressed data and the second part of the data;

[0201] Execution unit 1110 is used to perform rendering operations based on target rendering data.

[0202] In this embodiment, an acquisition unit acquires initial rendering data, which is used to render the game scene to be displayed within the graphical user interface of the mobile device; a partitioning unit divides the initial rendering data into a first part and a second part based on the data type, wherein the first part is compressible data and the second part is uncompressible data; a processing unit compresses the first part to obtain compressed data; a determining unit determines the target rendering data using the compressed data and the second part; and an execution unit performs a rendering operation based on the target rendering data. In other words, this invention...

[0203] It should be noted that the above-mentioned units can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above-mentioned units are located in the same processor; or, the above-mentioned units are located in different processors in any combination.

[0204] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0205] Optionally, in this embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0206] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0207] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0208] S1, Obtain initial rendering data, where the initial rendering data is used to render the game scene to be displayed in the graphical user interface of the mobile terminal;

[0209] S2, based on the data type of the initial rendering data, divide the initial rendering data into a first part of data and a second part of data, wherein the first part of data is the compressible data in the initial rendering data and the second part of data is the non-compressible data in the initial rendering data;

[0210] S3, compress the first part of the data to obtain compressed data;

[0211] S4, using compressed data and the second part of the data, determines the target rendering data;

[0212] S5 performs rendering operations based on the target rendering data.

[0213] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: dividing the first part of data into a first type of data and a second type of data, wherein the first type of data is used to characterize the corresponding normal data, and the second type of data is used to characterize the data in the first part of data other than the normal data; compressing the first type of data and the second type of data to obtain compressed data.

[0214] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: converting first type data into planar data to obtain first compressed data, wherein the number of channels in the first compressed data is less than the number of channels in the first type data.

[0215] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: determining the data to be saved in the second type of data based on the pixel position of the second type of data; saving the data to be saved to obtain second compressed data.

[0216] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: in response to a pixel position being a first position, determining the data content corresponding to the first position as data to be saved in a second type of data; in response to a pixel position being a second position, determining the data content corresponding to the second position as data to be saved in a second type of data.

[0217] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: performing a restoration process on the second compressed data to obtain rendering data corresponding to the second compressed data; and obtaining target rendering data based on the rendering data, the first compressed data, and the second part of the data.

[0218] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: determining, based on an objective function, compressed data horizontally adjacent to the second compressed data and compressed data vertically adjacent to the second compressed data, wherein the objective function is used to determine the difference between pixel values ​​corresponding to adjacent compressed data; and performing restoration processing on the second compressed data based on the horizontally adjacent compressed data and the vertically adjacent compressed data to obtain rendering data corresponding to the second compressed data.

[0219] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following steps: determining the difference between the luminance data in the second compressed data and the luminance data in the horizontally adjacent compressed data to obtain a first difference; determining the difference between the luminance data in the second compressed data and the luminance data in the vertically adjacent compressed data to obtain a second difference; and performing interpolation calculations on the horizontally adjacent compressed data and the vertically adjacent compressed data based on the first difference and the second difference to obtain rendering data corresponding to the second compressed data.

[0220] Optionally, the aforementioned computer-readable storage medium is further configured to store program code for performing the following step: the difference between the pixel value of the second compressed data and the pixel value of the compressed data at adjacent locations satisfies a pixel threshold.

[0221] In the computer-readable storage medium of this embodiment, a data processing technical solution is provided. By classifying the initial rendering data according to the data type, a first part of data and a second part of data are obtained. By compressing the first part of data, the initial rendering data is compressed, thereby increasing the data storage capacity and achieving the technical effect of improving the screen rendering performance, thus solving the technical problem of poor screen rendering performance.

[0222] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0223] In exemplary embodiments of this application, a computer-readable storage medium stores a program product capable of implementing the methods described above in this embodiment. In some possible implementations, various aspects of the embodiments of the present invention can also be implemented as a program product comprising program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this embodiment according to various exemplary embodiments of the present invention.

[0224] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In the embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0225] The aforementioned program product may take the form of any combination of one or more computer-readable media. Such computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (not exhaustive) of computer-readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0226] It should be noted that the program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0227] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0228] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0229] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0230] S1, Obtain initial rendering data, where the initial rendering data is used to render the game scene to be displayed in the graphical user interface of the mobile terminal;

[0231] S2, based on the data type of the initial rendering data, divide the initial rendering data into a first part of data and a second part of data, wherein the first part of data is the compressible data in the initial rendering data and the second part of data is the non-compressible data in the initial rendering data;

[0232] S3, compress the first part of the data to obtain compressed data;

[0233] S4, using compressed data and the second part of the data, determines the target rendering data;

[0234] S5 performs rendering operations based on the target rendering data.

[0235] Optionally, the processor may also be configured to perform the following steps via a computer program: dividing the first part of the data into a first type of data and a second type of data, wherein the first type of data is used to represent the corresponding normal data and the second type of data is used to represent the data in the first part of the data other than the normal data; and compressing the first type of data and the second type of data to obtain compressed data.

[0236] Optionally, the processor may also be configured to perform the following steps via a computer program: converting the first type of data into planar data to obtain first compressed data, wherein the number of channels in the first compressed data is less than the number of channels in the first type of data.

[0237] Optionally, the processor may also be configured to perform the following steps via a computer program: determining the data to be saved in the second type of data based on the pixel position of the second type of data; saving the data to be saved to obtain the second compressed data.

[0238] Optionally, the processor may also be configured to perform the following steps via a computer program: in response to a pixel position being a first position, determining the data content corresponding to the first position as data to be saved in a second type of data; in response to a pixel position being a second position, determining the data content corresponding to the second position as data to be saved in a second type of data.

[0239] Optionally, the processor may also be configured to perform the following steps via a computer program: perform restoration processing on the second compressed data to obtain rendering data corresponding to the second compressed data; and obtain target rendering data based on the rendering data, the first compressed data, and the second part of the data.

[0240] Optionally, the processor may also be configured to perform the following steps via a computer program: determining, based on an objective function, compressed data that is horizontally adjacent to the second compressed data and compressed data that is vertically adjacent to the second compressed data, wherein the objective function is used to determine the difference between pixel values ​​corresponding to adjacent compressed data; and performing restoration processing on the second compressed data based on the horizontally adjacent compressed data and the vertically adjacent compressed data to obtain rendering data corresponding to the second compressed data.

[0241] Optionally, the processor may also be configured to perform the following steps via a computer program: determining the difference between the luminance data in the second compressed data and the luminance data in the horizontally adjacent compressed data to obtain a first difference; determining the difference between the luminance data in the second compressed data and the luminance data in the vertically adjacent compressed data to obtain a second difference; and performing interpolation calculations on the horizontally adjacent compressed data and the vertically adjacent compressed data based on the first difference and the second difference to obtain the rendering data corresponding to the second compressed data.

[0242] Optionally, the processor may also be configured to perform the following steps via a computer program: the difference between the pixel value of the second compressed data and the pixel value of the compressed data at adjacent locations satisfies a pixel threshold.

[0243] In the electronic device of this embodiment, a data processing technical solution is provided. By classifying the initial rendering data according to the data type, a first part of data and a second part of data are obtained. By compressing the first part of data, the initial rendering data is compressed, thereby increasing the data storage capacity and achieving the technical effect of improving the screen rendering performance, thus solving the technical problem of poor screen rendering performance.

[0244] Figure 12 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Figure 12 As shown, the electronic device 1200 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0245] like Figure 12 As shown, the electronic device 1200 is presented in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processor 1210, at least one memory 1220, a bus 1230 connecting different system components (including memory 1220 and processor 1210), and a display 1240.

[0246] The memory 1220 stores program code that can be executed by the processor 1210, causing the processor 1210 to perform the steps described in the method section of the embodiments of this application according to various exemplary implementations of the present invention.

[0247] The memory 1220 may include a readable medium in the form of volatile memory cells, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include read-only memory (ROM) 12203, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0248] In some instances, memory 1220 may also include programs / utilities 12204 having a set (at least one) of program modules 12205, including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Memory 1220 may further include memory remotely located relative to processor 1210, which can be connected to electronic device 1200 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0249] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, peripheral bus, graphics acceleration port, processor 1210, or a local bus using any of the various bus structures.

[0250] The display 1240 may be, for example, a touch screen liquid crystal display (LCD) that allows a user to interact with the user interface of the electronic device 1200.

[0251] Optionally, the electronic device 1200 can also communicate with one or more external devices 1400 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 1200, and / or any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via the input / output (I / O) interface 1250. Furthermore, the electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via the network adapter 1260. Figure 12 As shown, network adapter 1260 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although... Figure 12 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0252] The aforementioned electronic device 1200 may further include: a keyboard, a cursor control device (such as a mouse), an input / output interface (I / O interface), a network interface, a power supply, and / or a camera.

[0253] Those skilled in the art will understand that Figure 12 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 1200 may also include components that are more... Figure 12 The more or fewer components shown, or having the same Figure 1 Different configurations are shown. The memory 1220 can be used to store computer programs and corresponding data, such as the computer program and corresponding data corresponding to the data processing method in this embodiment of the invention. The processor 1210 executes various functional applications and data processing by running the computer program stored in the memory 1220, thereby implementing the aforementioned data processing method.

[0254] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0255] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0256] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0257] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0258] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0259] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0260] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that, include: Obtain initial rendering data, wherein the initial rendering data is used to render the game scene to be displayed in the graphical user interface of the mobile terminal; Based on the data type of the initial rendering data, the initial rendering data is divided into a first part of data and a second part of data, wherein the first part of data is compressible data in the initial rendering data, and the second part of data is non-compressible data in the initial rendering data; The first part of the data is compressed to obtain compressed data; Using the compressed data and the second part of the data, the target rendering data is determined; Perform rendering operations based on the target rendering data; The step of compressing the first part of the data to obtain compressed data includes: dividing the first part of the data into a first type of data and a second type of data, wherein the first type of data is used to represent the corresponding normal data, and the second type of data is used to represent the data in the first part of the data other than the normal data; and compressing the first type of data and the second type of data to obtain the compressed data. The step of compressing the second type of data to obtain the compressed data includes: determining the data to be saved in the second type of data based on the pixel position of the second type of data; and saving the data to be saved to obtain the second compressed data. The step of determining the target rendering data using the compressed data and the second part of the data includes: performing a restoration process on the second compressed data to obtain the rendering data corresponding to the second compressed data; and obtaining the target rendering data based on the rendering data, the first compressed data, and the second part of the data.

2. The method according to claim 1, characterized in that, The first type of data is compressed to obtain the compressed data, including: The first type of data is converted into planar data to obtain first compressed data, wherein the number of channels in the first compressed data is less than the number of channels in the first type of data.

3. The method according to claim 1, characterized in that, Based on the pixel positions of the second type of data, determine the data to be saved in the second type of data, including: In response to the pixel position being a first position, the data content corresponding to the first position is determined as the data to be saved in the second type of data; In response to the pixel position being the second position, the data content corresponding to the second position is determined as the data to be saved in the second type of data.

4. The method according to claim 1, characterized in that, The second compressed data is decompressed to obtain the corresponding rendering data, including: Based on the objective function, the compressed data that is horizontally adjacent to the second compressed data and the compressed data that is vertically adjacent to the second compressed data are determined, wherein the objective function is used to determine the difference between the pixel values ​​corresponding to the adjacent compressed data; Based on the horizontally adjacent compressed data and the vertically adjacent compressed data, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data.

5. The method according to claim 4, characterized in that, Based on the horizontally adjacent compressed data and the vertically adjacent compressed data, the second compressed data is restored to obtain the rendering data corresponding to the second compressed data, including: The difference between the brightness data in the second compressed data and the brightness data in the horizontally adjacent compressed data is determined to obtain the first difference; The difference between the brightness data in the second compressed data and the brightness data in the compressed data adjacent in the vertical direction is determined to obtain the second difference; Based on the first difference and the second difference, interpolation calculations are performed on the horizontally adjacent compressed data and the vertically adjacent compressed data to obtain the rendering data corresponding to the second compressed data.

6. The method according to any one of claims 1 to 5, characterized in that, The difference between the pixel value of the second compressed data and the pixel value of the compressed data at the adjacent position satisfies the pixel threshold.

7. A data processing apparatus, characterized in that, include: An acquisition unit is used to acquire initial rendering data, wherein the initial rendering data is used to render the game scene to be displayed in the graphical user interface of the mobile terminal; A partitioning unit is used to divide the initial rendering data into a first part of data and a second part of data based on the data type of the initial rendering data, wherein the first part of data is compressible data in the initial rendering data and the second part of data is non-compressible data in the initial rendering data; The processing unit is used to compress the first part of the data to obtain compressed data; A determining unit is used to determine target rendering data using the compressed data and the second part of the data; An execution unit is used to perform rendering operations based on the target rendering data; The processing unit is configured to compress the first part of the data through the following steps to obtain the compressed data: dividing the first part of the data into a first type of data and a second type of data, wherein the first type of data is used to represent the corresponding normal data, and the second type of data is used to represent the data in the first part of the data other than the normal data; and compressing the first type of data and the second type of data to obtain the compressed data. The processing unit is further configured to compress the second type of data through the following steps to obtain the compressed data: determining the data to be saved in the second type of data based on the pixel position of the second type of data; saving the data to be saved to obtain the second compressed data; The determining unit is further configured to determine the target rendering data by using the compressed data and the second part of the data through the following steps: performing restoration processing on the second compressed data to obtain rendering data corresponding to the second compressed data; and obtaining the target rendering data based on the rendering data, the first compressed data, and the second part of the data.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method described in any one of claims 1 to 6 when run by a processor.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 6.

Images