Image data processing method and device, graphics processor, medium and equipment

Abstract

The present disclosure relates to an image data processing method and device, a graphics processor, a medium and equipment. The method is used for a graphics processor, comprising: obtaining original image data of a target image, and storing the original image data to a storage buffer of the graphics processor, the original image data being in a first format, the first format being a format in which multi-bit data of a pixel is arranged in different bit planes in layers; performing data processing on a plurality of pixels of the target image to obtain pixel conversion data of each of the plurality of pixels; wherein the data processing on one pixel comprises: obtaining a plurality of valid bit data of the pixel from the storage buffer using a mask, and combining the plurality of valid bit data of the pixel into pixel conversion data of the pixel; and obtaining target image data conforming to a second format according to the pixel conversion data of each pixel of the target image, the second format being different from the first format.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to an image data processing method and device, a graphics processor, a medium and equipment. BACKGROUND

[0002] In the field of computer graphics processing, the organization form of image data is usually to divide multiple bit planes according to multiple color components of pixels and store them in layers. In actual application scenarios, the rendering of images usually cannot be directly implemented based on this format, and needs to be converted into a format suitable for image rendering. At present, the above-mentioned format conversion process has the problem of low efficiency, especially when processing the conversion of high-resolution images, obvious delay is caused due to low conversion efficiency, which is difficult to meet the requirements of real-time rendering. SUMMARY

[0003] The purpose of the present disclosure is to provide an image data processing method, device, graphics processor, medium and equipment.

[0004] In order to achieve the above-mentioned purpose, according to a first aspect of the present disclosure, an image data processing method is provided, which is used for a graphics processor, and the method comprises:

[0005] Obtaining original image data of a target image and storing the original image data into a storage buffer of the graphics processor, the original image data being in a first format, the first format being a format in which multiple-bit data of a pixel is arranged in layers according to different bit planes;

[0006] Performing data processing on multiple pixels of the target image to obtain pixel conversion data of each of the multiple pixels; wherein the data processing on one pixel comprises: obtaining multiple valid bit data of the pixel from the storage buffer by using a mask, and combining the multiple valid bit data of the pixel into pixel conversion data of the pixel;

[0007] Obtaining target image data conforming to a second format according to the pixel conversion data of each pixel in the target image, the second format being different from the first format.

[0008] Optionally, the original image data corresponds to multiple data layers, and one data layer stores bit data of all pixels in the target image at a specified bit plane;

[0009] The obtaining of the multiple valid bit data of the pixel from the storage buffer by using the mask comprises:

[0010] According to a target index of the pixel in the storage buffer, determining multiple data layers corresponding to the pixel as target data layers;

[0011] According to the target index, a binary bit mask is generated;

[0012] According to the binary bit mask, logical AND operation is performed on data of each target data layer respectively to obtain the multiple valid bit data of the pixel.

[0013] Optionally, the generating of the binary bit mask according to the target index comprises:

[0014] According to the target index, a bit offset of the pixel in the storage buffer is determined;

[0015] According to the bit offset, the binary bit mask is generated.

[0016] Optionally, the second format corresponds to a byte order, and the byte order is used to represent an arrangement order of color channels, wherein one byte corresponds to multiple bit data of one color channel.

[0017] The combining of the multiple valid bit data of the pixel into the pixel conversion data of the pixel comprises:

[0018] According to an arrangement order of the target data layers in the original image data, the multiple valid bit data are combined into initial splicing data;

[0019] In response to determining that a byte order corresponding to the initial splicing data is inconsistent with a byte order corresponding to the second format, the byte order of the initial splicing data is adjusted to obtain pixel conversion data conforming to the byte order of the second format;

[0020] In response to determining that the byte order corresponding to the initial splicing data is consistent with the byte order corresponding to the second format, the initial splicing data is determined as the pixel conversion data.

[0021] Optionally, the storage buffer comprises a shader storage buffer object.

[0022] The shader storage buffer object is generated by:

[0023] Configuration information for the shader storage buffer object is obtained, and the configuration information comprises a memory layout rule;

[0024] Attribute information of the shader storage buffer object is determined, and the attribute information at least comprises the memory layout rule;

[0025] According to the attribute information, the shader storage buffer object is created in the graphic processor.

[0026] Optionally, the attribute information further comprises a life cycle type of the shader storage buffer object, and / or a data storage capacity; the life cycle type comprises a temporary type and a persistent type.

[0027] If the attribute information comprises the life cycle type, the determining the attribute information of the shader storage buffer object comprises:

[0028] acquiring a frequency of image processing by the graphics processor;

[0029] If the frequency is less than or equal to a preset threshold, determining the life cycle type as the temporary type;

[0030] If the frequency is greater than the preset threshold, determining the life cycle type as the persistent type;

[0031] If the attribute information comprises the data storage capacity, the determining the attribute information of the shader storage buffer object comprises:

[0032] determining the data storage capacity according to an image width of the target image, an image height of the target image, and a bit depth of the target image.

[0033] Optionally, the data processing on the plurality of pixels of the target image to obtain pixel conversion data of each of the plurality of pixels comprises:

[0034] acquiring, by a fragment shader of the graphics processor, original image data of the target image from the storage buffer;

[0035] allocating, based on the original image data, an independent thread for each pixel of the target image, so that each thread performs the data processing on one pixel in parallel, and each thread outputs pixel conversion data of one pixel.

[0036] Optionally, in the second format, bit data of a single pixel is stored continuously, and data of adjacent pixels is arranged continuously.

[0037] The obtaining the target image data in the second format according to the pixel conversion data of each pixel of the target image comprises:

[0038] sequentially storing the pixel conversion data output by each thread according to coordinates of the pixels in the target image, to obtain the target image data in the second format.

[0039] Optionally, the method further comprises:

[0040] determining color output values corresponding to the pixels in the target image according to the target image data;

[0041] inputting the color output value into a frame buffer object to render and output the frame buffer object.

[0042] According to a second aspect of the present disclosure, there is provided an image data processing apparatus for a graphics processor, the apparatus comprising:

[0043] a first obtaining module configured to obtain original image data of a target image and store the original image data into a storage buffer of the graphics processor, the original image data being in a first format, the first format being a format in which multi-bit data of a pixel is arranged in different bit planes in layers;

[0044] a processing module configured to perform data processing on a plurality of pixels of the target image to obtain pixel conversion data of each of the plurality of pixels, wherein the data processing on one pixel comprises: obtaining a plurality of valid bit data of the pixel from the storage buffer using a mask, and combining the plurality of valid bit data of the pixel into the pixel conversion data of the pixel;

[0045] a generating module configured to obtain target image data in a second format according to the pixel conversion data of each pixel of the target image, the second format being different from the first format.

[0046] According to a third aspect of the present disclosure, there is provided a graphics processor, which, when executed by a processor, implements the steps of the method according to the first aspect of the present disclosure.

[0047] According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method according to the first aspect of the present disclosure.

[0048] According to a fifth aspect of the present disclosure, there is provided an electronic device comprising:

[0049] a memory having stored thereon a computer program;

[0050] a processor configured to execute the computer program in the memory to implement the steps of the method according to the first aspect of the present disclosure.

[0051] According to a sixth aspect of the present disclosure, there is provided a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to the first aspect of the present disclosure.

[0052] By the technical solution, the original image data of the target image is acquired, and the original image data is stored in the storage buffer of the graphics processor. The original image data is in a first format in which pixel multi-bit data is arranged in different bit planes. By storing the original image data in the first format in the storage buffer of the graphics processor, the original image data can be directly accessed. Compared with storing the original image data in the central processing unit, the performance loss caused by frequent cross-memory boundary access to image data in different bit planes can be effectively avoided. Moreover, the data of a plurality of pixels of the target image is processed to obtain pixel conversion data of each of the plurality of pixels. Then, the target image data in a second format is obtained according to the pixel conversion data of each pixel in the target image. The data processing for one pixel includes: obtaining multi-effective-bit data of the pixel from the storage buffer by using a mask, and combining the multi-effective-bit data of the pixel into the pixel conversion data of the pixel. In this way, the effective data of each pixel scattered in different bit planes is extracted from the storage buffer by using the mask mechanism, and is combined into the pixel conversion data of each pixel. The data of the same pixel scattered in different bit planes is accurately filtered out and recombined into the complete pixel conversion data of the pixel. Then, the target image data in the second format different from the first format is obtained according to the pixel conversion data of each pixel, which is beneficial to improving the conversion efficiency. In this way, the memory access efficiency and the data processing efficiency of the image data format conversion can be improved as a whole, the processing delay is effectively reduced, and the real-time rendering and high-performance graphics processing are guaranteed.

[0053] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, and are used to explain the present disclosure together with the following detailed description. In the drawings:

[0055] Figure 1 is a flowchart of an image data processing method according to an embodiment of the present disclosure;

[0056] Figure 2 is an exemplary schematic diagram of a data arrangement manner in a first format according to an embodiment of the present disclosure;

[0057] Figure 3 is an exemplary schematic diagram of a data arrangement manner in a second format according to an embodiment of the present disclosure;

[0058] Figure 4 is a block diagram of an image data processing device according to an embodiment of the present disclosure;

[0059] Figure 5 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] The specific embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to illustrate and explain the present disclosure, and are not intended to limit the present disclosure.

[0061] As described in the background, the image data in the format of multiple color components of pixels divided into multiple planes for layered storage cannot meet the requirements of image rendering, and generally needs to be converted in format. In the related art, the image data is generally converted in format by a CPU (Central Processing Unit), and the converted data is provided to a GPU (Graphics Processing Unit) for subsequent image rendering to better adapt to the parallel processing mechanism of the GPU. However, based on the above layered storage structure, the CPU needs to perform a large amount of layer-by-layer traversal and bit-by-bit processing in the processing process, resulting in slow calculation speed and low conversion efficiency, which leads to the fact that high-efficiency real-time conversion cannot be achieved when facing large-scale image data processing. Therefore, the current image format conversion based on the CPU has the problems of insufficient conversion efficiency and inability to achieve high-efficiency real-time conversion.

[0062] To solve the above technical problems, the present disclosure provides an image data processing method, device, graphics processor, medium and equipment to improve the conversion efficiency and thus meet the requirements of real-time rendering and high-performance graphics processing.

[0063] Figure 1 is a flowchart of an image data processing method according to an embodiment of the present disclosure. The method provided by the present disclosure can be applied to a graphics processor (GPU). As shown in Figure 1 The image data processing method provided by the present disclosure can include steps 11 to 13.

[0064] In step 11, the original image data of a target image is obtained, and the original image data is stored in a storage buffer of a graphics processor, the original image data being in a first format, and the first format being a format in which multiple-bit data of a pixel is arranged in different bit planes in layers.

[0065] Generally, a pixel is composed of multiple-bit data, and the bit data is allocated according to color components, and a specific color is obtained by mixing color component values. For example, in a 24-bit pixel, red (R), green (G), and blue (B) each occupy 8 bits. For another example, in a 32-bit pixel, red (R), green (G), blue (B), and transparency (Alpha) each occupy 8 bits.

[0066] The original image data is in a first format, in which multi-bit data of a pixel is arranged in different bit planes, for example, if the pixel is 24-bit, it is divided into 24 data layers, i.e., 24 bit planes. Each data layer corresponds to one bit of the pixel, and each data layer contains bit data of all pixels in the target image at a specified bit plane. The arrangement of the bit data of each pixel in each data layer is consistent with the arrangement of each pixel in the target image. For example, assuming that the pixel includes 4-bit data, the arrangement of the data in the first format can be as shown in Table 1. Figure 2

[0067] The graphics processor can obtain the original image data of the target image and store the original image data in a storage buffer of the graphics processor. The storage buffer can be a space that has been generated and used to store the original image data of the image to be processed (i.e., the target image). Alternatively, the storage buffer can be a space that can be directly accessed by the shader, such as a SSBO (Shader Storage Buffer Object), a UBO (Uniform Buffer Object), etc., to improve the processing efficiency of the subsequent processing steps.

[0068] In one possible implementation, the storage buffer can include a shader storage buffer object (SSBO) that can be directly read and written by the shader, which can greatly improve the efficiency of subsequent data processing. Accordingly, the shader storage buffer object can be generated in the following manner:

[0069] obtaining configuration information for the shader storage buffer object, the configuration information including a memory layout rule;

[0070] determining attribute information of the shader storage buffer object, the attribute information including at least the memory layout rule;

[0071] creating the shader storage buffer object in the graphics processor according to the attribute information.

[0072] ​Optionally, the user can set the configuration information according to actual needs, or a set of default configuration information can be pre-set. When a shader storage buffer object needs to be generated, the configuration information for the shader storage buffer object can be obtained to guide the generation of the shader storage buffer object as attribute information of the shader storage buffer object. The configuration information can include a memory layout rule, which can be set when defining the structure of the shader storage buffer object (SSBO). For example, the memory layout rule can include std140 layout. For another example, the memory layout rule can include std430 layout, which optimizes the data storage mode of the shader storage buffer object (SSBO), effectively reduces the memory reading overhead of the shader, and further greatly improves the efficiency of data access.

[0073] By determining the attribute information of the shader storage buffer object, a corresponding shader storage buffer object can be created in the graphics processor according to the attribute information. The attribute information can at least include the memory layout rule described above. Optionally, the attribute information of the shader storage buffer object can further include the life cycle type of the shader storage buffer object and / or the data storage capacity. The life cycle type can include temporary type and persistent type, and the life cycle type of temporary type represents a shorter life cycle, for example, generated when raw image data needs to be stored, and released after the raw image data is processed; the life cycle type of persistent type represents a longer life cycle, for example, pre-generated and persistently existing in the graphics processor GPU.

[0074] Optionally, if the attribute information includes the life cycle type, the life cycle type in the attribute information can be determined in the following manner:

[0075] Obtain the frequency at which the graphics processor processes images;

[0076] If the frequency is less than or equal to a preset threshold, the life cycle type is determined to be temporary type;

[0077] If the frequency is greater than the preset threshold, the life cycle type is determined to be persistent type.

[0078] For example, the frequency at which the graphics processor processes images can be determined according to the number of images processed or to be processed by the graphics processor in a specified period. Based on this, if the frequency at which the graphics processor processes images is less than or equal to a preset threshold, the life cycle type can be determined to be temporary type, and if the frequency at which the graphics processor processes images is greater than the preset threshold, the life cycle type can be determined to be persistent type. The preset threshold can be flexibly set according to actual needs.

[0079] Optionally, if the attribute information includes the data storage capacity, the data storage capacity in the attribute information can be determined in the following manner:

[0080] The data storage capacity is determined according to the image width of the target image, the image height of the target image, and the bit depth of the target image.

[0081] For example, according to the image width of the target image, the image height of the target image, and the bit depth of the target image, the capacity of the minimum space capable of accommodating the target image can be determined, and the data storage capacity can be set to a value not less than the capacity of the minimum space.

[0082] After determining the attribute information of the shader storage buffer object in the above manner, the shader storage buffer object can be created in the graphics processor based on the attribute information.

[0083] In the above manner, the shader storage buffer object is created in the graphics processor to store the original image data of the target image to be processed, so that the subsequent shader can directly read and write the original image data through the shader storage buffer object, the data reading and storage mode is optimized, and the processing efficiency of the graphics processor is improved.

[0084] In step 12, the data of the plurality of pixels of the target image is processed to obtain pixel conversion data of each of the plurality of pixels; wherein the data processing for one pixel includes: obtaining a plurality of valid bit data of the pixel from the storage buffer using a mask, and combining the plurality of valid bit data of the pixel into pixel conversion data of the pixel.

[0085] Optionally, the data of the plurality of pixels of the target image is processed to obtain pixel conversion data of each of the plurality of pixels, which can be implemented in the following manner:

[0086] The original image data of the target image is obtained from the storage buffer by the fragment shader of the graphics processor;

[0087] Based on the original image data, an independent thread is allocated for each pixel of the target image, so that each thread performs data processing for one pixel in parallel, and each thread outputs pixel conversion data of one pixel.

[0088] The fragment shader of the graphics processor (GPU) can retrieve the raw image data of the target image from the storage buffer. The fragment shader can also be called a pixel shader. The graphics rendering pipeline is a series of processing stages in which the GPU performs graphics processing and generates the final screen image. It mainly includes a vertex processing stage, which uses a vertex shader to map the vertex data of the 3D model to screen space; a rasterization stage, which connects vertices into geometric primitives and converts them into pixel fragments; a pixel shading stage, which uses a fragment shader to calculate the color of each pixel; and a frame buffer stage, which writes the processed pixel data to the frame buffer and outputs it to the display screen. In this disclosure, the relevant steps up to the rasterization stage can refer to commonly used vertex processing and rasterization methods, and will not be elaborated here.

[0089] After acquiring the raw image data, an independent thread can be allocated to each pixel of the target image to process the data of each pixel separately, thereby achieving parallel processing of multiple pixels and significantly improving data processing efficiency. Data processing for a single pixel may include the following steps:

[0090] The mask is used to obtain multiple valid bits of pixel data from the storage buffer, and the multiple valid bits of pixel data are combined into pixel transformation data.

[0091] As mentioned earlier, the raw image data corresponds to multiple data layers, and each data layer can store the bit data of all pixels in the target image in a specified bit plane. Accordingly, retrieving multiple valid bit data of a pixel from the storage buffer using a mask can include the following steps:

[0092] Based on the target index of the pixel in the storage buffer, determine the multiple data layers corresponding to the pixel, which are then used as the target data layers;

[0093] Generate a binary bitmask based on the target index;

[0094] Based on the binary bitmask, a logical AND operation is performed with the data of each target data layer to obtain multiple valid bits of pixel data.

[0095] Alternatively, by pixels in the fragment shader gl_FragCoord The parameters determine the coordinates of the pixel on the screen. x , y Furthermore, considering the screen coordinates rendered by the graphics processor and the origin of the first format, y Differences in axis direction (screen coordinates default to the bottom left corner of the screen as the origin, y Axial axis upwards, the first format defaults to the top left corner of the screen as the origin. y (axis downward), screen coordinates (x , y ) to correspond to the format of the original image data, and the converted y’ = h - y , and the pixel converted coordinate (x', y') can be obtained based on this, wherein x' is the x-coordinate of the pixel converted coordinate, y' is the y-coordinate of the pixel converted coordinate, height is the height of the screen, and x is the x-coordinate of the pixel in the original image data, y is the y-coordinate of the pixel in the original image data. x , y’ ) can be calculated based on the pixel converted coordinate (x', y'), and thus the target index can be determined. h x , y’ ) can be calculated based on the pixel converted coordinate (x', y'), and thus the target index can be determined.

[0096] After that, according to the target index, the data layer in which the bits of the pixel are located can be located as the target data layer. Under the bit data arrangement mechanism of the first format, the target data layer not only contains the bit data of the pixel currently processed, but also includes the bit data of other pixels in the target image, and therefore, all bit data contained in the target data layer needs to be screened to extract the valid bit data of the currently processed pixel.

[0097] The extraction of the above-mentioned valid bit data can be realized by a binary bit mask. According to the target index, a binary bit mask for data screening can be generated to retain the bit data corresponding to the currently processed pixel through the binary bit mask and ignore the bit data of other pixels irrelevant to the currently processed pixel.

[0098] Optionally, generating the binary bit mask according to the target index can include the following steps:

[0099] According to the target index, determining the bit offset of the pixel in the storage buffer;

[0100] According to the bit offset, generating the binary bit mask.

[0101] Optionally, the bit offset of the pixel in the storage buffer can be determined in the following manner:

[0102] Using the target index, performing a modulo operation on the number of bits corresponding to the pixel, and taking the result of the modulo operation as the bit offset.

[0103] For example, the bit offset can be determined in the following manner: o :

[0104] o = gid.x % perPixelBits

[0105] wherein, gid.x is the target index, % is the modulo symbol, and perPixelBits is the total number of bits of the pixel.​

[0106] After obtaining the bit offset, a binary bit mask can be generated according to the bit offset. In generating the binary bit mask, the bit corresponding to the offset is usually set as a valid bit 1, and other bits are set as invalid bits 0. For example, assuming that the offset obtained is 3, the bit mask 0x00001000 can be obtained by setting the bit 3 bits to the left of the least significant bit (bit 0) as a valid bit 1 and setting other bits as 0.

[0107] Since the positions of the pixels in each layer are the same under the arrangement mechanism of the first format, i.e., the corresponding positions in each target data layer are the same, the binary bit mask generated in the above manner can be used to perform logical AND operation on the data corresponding to each target data layer respectively by using the binary bit mask, to retain the bit data of the valid bits and delete the bit data of the invalid bits, so as to perform data screening, and a plurality of retained bit data can be obtained, which are the valid bit data of the pixel currently processed.

[0108] The first format and the second format usually correspond to a byte order, which is used to represent the arrangement order of the color channels, wherein one byte can correspond to a plurality of bit data of one color channel. Here, the byte represents the color channel, i.e., R, G, B, and A as described above. The byte order is arranged based on a setting, for example, can be set as RGBA or ARGB, etc. For example, for RGBA, if each color channel corresponds to 8 bits, in a 32-bit pixel, from left to right, they will be arranged as 8 bits of R, 8 bits of G, 8 bits of B, and 8 bits of A.

[0109] After obtaining a plurality of valid bit data of a pixel, the plurality of valid bit data of the pixel can be combined into pixel conversion data of the pixel, and the pixel conversion data contains complete bit data of the pixel.

[0110] In a possible implementation, combining the plurality of valid bit data of the pixel into the pixel conversion data of the pixel can include the following steps:

[0111] combining the plurality of valid bit data into initial splicing data according to the arrangement order of the target data layers in the original image data;

[0112] in response to determining that the byte order corresponding to the initial splicing data is inconsistent with the byte order corresponding to the second format, adjusting the byte order of the initial splicing data to obtain the pixel conversion data conforming to the byte order of the second format;

[0113] in response to determining that the byte order corresponding to the initial splicing data is consistent with the byte order corresponding to the second format, determining the initial splicing data as the pixel conversion data.

[0114] According to the arrangement order of the target data layers in the original image data, that is, the byte order corresponding to the first format, the multiple valid bit data can be combined into initial splicing data.

[0115] In the case where the byte orders of the first format and the second format are different, the arrangement mode of the bit data of the pixels in the initial splicing data is not the same as the bit data arrangement mode of the second format, and needs to be adjusted to be consistent with the second format, so as to ensure the accuracy of subsequent image rendering. Therefore, in the case where the byte order corresponding to the initial splicing data is inconsistent with the byte order corresponding to the second format, in response thereto, the byte order of the initial splicing data can be adjusted to obtain target image data conforming to the byte order of the second format; in the case where the byte order corresponding to the initial splicing data is consistent with the byte order corresponding to the second format, in response thereto, the initial splicing data can be directly determined as the target image data.

[0116] For example, if the byte orders of the first format and the second format are ARGB and ABGR respectively, that is, the byte orders are inconsistent, and each color channel corresponds to 8-bit data, the pixel conversion data can be determined according to the following code packedInt :

[0117] uint part 1 = ( result &0x000000FFu)<<16;

[0118] uint part 2 = ( result &0x00FF0000u)>>16;

[0119] packedInt = ( result &~0x00FF00FFu) | part 1 | part 2。

[0120] In the above manner, the 0th-7th bits and the 16th-23rd bits of the initial splicing data can be exchanged to realize the conversion of the byte orders from ARGB to ABGR.

[0121] In the above manner, by using the data screening mechanism of the binary bit mask and the combination of the screened bit data, the multiple valid bit data corresponding to the pixels can be accurately screened, and can be spliced according to the byte order conforming to the second format, so as to obtain the pixel conversion data corresponding to each pixel respectively, and the data processing efficiency is high.

[0122] Referring to the above-mentioned data processing mode of a pixel, under the multi-thread parallel processing mechanism of the graphics processor, each thread can quickly process a single pixel and output pixel conversion data corresponding to the pixel, so that pixel conversion data corresponding to each pixel can be quickly obtained.

[0123] In step 13, target image data conforming to a second format is obtained according to the pixel conversion data of each pixel in the target image, and the second format is different from the first format.

[0124] Optionally, in the second format, the bit data of a single pixel is stored continuously, and the data of adjacent pixels is arranged continuously. For example, assuming that a pixel includes 4-bit data, the data arrangement mode of the second format can be as shown in Figure 3 , in which the bit data of pixel A1 is arranged in sequence, the bit data of pixel A2 is arranged in sequence, and so on. For example, the second format can be a ZPixmap format. Accordingly, step 13 can include the following steps.

[0125] According to the coordinates of the pixels in the target image, the pixel conversion data output by each thread is sequentially and orderly stored to obtain target image data conforming to the second format.

[0126] That is, according to the coordinates of each pixel in the target image, the complete pixel conversion data of each pixel is sequentially stored according to the arrangement order of the coordinates, so that target image data conforming to the second format can be obtained.

[0127] In a possible implementation, based on the steps shown in Figure 1 , the method provided by the present disclosure can further include the following steps.

[0128] According to the target image data, color output values corresponding to the pixels in the target image are determined;

[0129] The color output values are input into a frame buffer object to render and output the frame buffer object.

[0130] For example, the color output values fragColor may be determined by the following code:

[0131] fragColor = vec 4(

[0132] float ( packedInt &0xFFu) / 255.0,

[0133] float (( packedInt >>8)&0xFFu) / 255.0,

[0134] fl o at (( packedInt >>16)&0xFFu) / 255.0,

[0135] float (( packedInt >>24)&0xFFu) / 255.0 )

[0137] Then, the color output values ​​are input into a frame buffer object (FBO) to render and output the frame buffer object. Specifically, by creating and binding the frame buffer object, a texture is attached to the color appendage of the frame buffer object, and after the pixel shader has processed all pixels in the target image, the resulting image in the frame buffer object is rendered to the display screen.

[0138] The above technical solution acquires the original image data of the target image and stores it in the memory buffer of the graphics processor. The original image data is a first format in which multi-bit data of pixels are arranged in layers according to different bit planes. By storing the original image data in the memory buffer of the graphics processor, the original image data can be accessed directly. Compared with storing the original image data in the central processing unit, this effectively avoids the performance loss caused by frequently accessing image data across memory boundaries. Furthermore, data processing is performed on multiple pixels of the target image to obtain pixel transformation data for each pixel. Then, based on the pixel transformation data of each pixel in the target image, target image data conforming to a second format is obtained. The data processing for a single pixel includes: obtaining multiple valid bit data of the pixel from the memory buffer using a mask, and combining the multiple valid bit data of the pixel into pixel transformation data for that pixel. Therefore, by employing a masking mechanism to extract the effective data of each pixel scattered across different bit planes from the storage buffer and combining it into pixel conversion data for each pixel, it is equivalent to accurately filtering out the data of the same pixel scattered across different bit planes and recombining it into complete pixel conversion data for that pixel. Furthermore, based on the pixel conversion data of each pixel, a second format of target image data, different from the first format, is obtained, which helps improve conversion efficiency. In this way, the memory access efficiency and data processing efficiency of image data format conversion can be improved overall, effectively reducing processing latency and providing a guarantee for real-time rendering and high-performance graphics processing.

[0139] Figure 4 This is a block diagram of an image data processing apparatus according to one embodiment of the present disclosure. The image data processing apparatus 40 can be applied to a graphics processor. Figure 4 As shown, the device 40 may include:

[0140] The first obtaining module 41 is configured to obtain original image data of a target image and store the original image data into a storage buffer of the graphics processor, the original image data being in a first format, and the first format being a format in which multi-bit data of a pixel is arranged in different bit planes in layers;

[0141] The processing module 42 is configured to perform data processing on a plurality of pixels of the target image to obtain pixel conversion data of each of the plurality of pixels, wherein the data processing on one pixel comprises: obtaining, by using a mask, a plurality of valid bit data of the pixel from the storage buffer, and combining the plurality of valid bit data of the pixel into pixel conversion data of the pixel;

[0142] The generating module 43 is configured to obtain target image data in a second format according to the pixel conversion data of each pixel in the target image, the second format being different from the first format.

[0143] Optionally, the original image data corresponds to a plurality of data layers, and one data layer stores bit data of all pixels in the target image in a specified bit plane.

[0144] The processing module 42 performs data processing on one pixel by using the following sub-modules:

[0145] The first determining sub-module is configured to determine, according to a target index of the pixel in the storage buffer, a plurality of data layers corresponding to the pixel as target data layers.

[0146] The first generating sub-module is configured to generate a binary bit mask according to the target index.

[0147] The operation sub-module is configured to perform logical AND operation on data of each of the target data layers respectively according to the binary bit mask to obtain a plurality of valid bit data of the pixel.

[0148] Optionally, the first generating sub-module comprises:

[0149] The second determining sub-module is configured to determine, according to the target index, a bit offset of the pixel in the storage buffer.

[0150] The second generating sub-module is configured to generate the binary bit mask according to the bit offset.

[0151] Optionally, the second format corresponds to a byte order, and the byte order is used to represent an arrangement order of color channels, wherein one byte corresponds to multi-bit data of one color channel.

[0152] The processing module 42 performs data processing on one pixel by using the following sub-modules:

[0153] a combination sub-module, configured to combine the plurality of valid bit data into initial stitching data according to an arrangement order of the original image data corresponding to each of the target data layers;

[0154] an adjustment sub-module, configured to adjust a byte order of the initial stitching data to obtain pixel conversion data conforming to a byte order of the second format, in response to determining that the byte order of the initial stitching data is inconsistent with the byte order of the second format;

[0155] a third determination sub-module, configured to determine the initial stitching data as the pixel conversion data, in response to determining that the byte order of the initial stitching data is consistent with the byte order of the second format.

[0156] Optionally, the storage buffer includes a shader storage buffer object.

[0157] The shader storage buffer object is generated by the following modules:

[0158] a second acquisition module, configured to acquire configuration information for the shader storage buffer object, the configuration information including a memory layout rule;

[0159] a first determination module, configured to determine attribute information of the shader storage buffer object, the attribute information at least including the memory layout rule;

[0160] a creation module, configured to create the shader storage buffer object in the graphics processor according to the attribute information.

[0161] Optionally, the attribute information further includes a life cycle type of the shader storage buffer object and / or a data storage capacity; the life cycle type includes a temporary type and a persistent type.

[0162] The first determination module includes:

[0163] an acquisition sub-module, configured to acquire a frequency of image processing by the graphics processor, if the attribute information includes the life cycle type.

[0164] a fourth determination sub-module, configured to determine the life cycle type as the temporary type, if the frequency is less than or equal to a preset threshold.

[0165] a fifth determination sub-module, configured to determine the life cycle type as the persistent type, if the frequency is greater than the preset threshold.

[0166] The sixth determining sub-module is configured to determine the data storage capacity according to the image width of the target image, the image height of the target image, and the bit depth of the target image if the attribute information comprises the data storage capacity.

[0167] Optionally, the processing module 42 is configured to: acquire original image data of the target image from the storage buffer by a fragment shader of the graphic processor; and assign an independent thread to each pixel of the target image based on the original image data, so that each thread executes the data processing for one pixel in parallel, and each thread outputs pixel conversion data of one pixel respectively.

[0168] Optionally, in the second format, the bit data of a single pixel is stored continuously, and the data of adjacent pixels is arranged continuously.

[0169] The generating module 43 is configured to sequentially store the pixel conversion data output by each thread in order according to the coordinates of the pixels in the target image, so as to obtain the target image data conforming to the second format.

[0170] Optionally, the device 40 further comprises:

[0171] The second determining module is configured to determine color output values corresponding to the pixels in the target image according to the target image data.

[0172] The output module is configured to input the color output values into a frame buffer object, so as to render and output the frame buffer object.

[0173] As to the device in the above embodiments, the specific manners in which the modules perform operations have been described in detail in the embodiments of the method, and thus will not be described here in detail.

[0174] Based on the same inventive concept, the disclosure further provides a graphic processor configured to execute the image data processing method provided in any of the embodiments of the disclosure.

[0175] Based on the same inventive concept, the disclosure further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the image data processing method provided in any of the embodiments of the disclosure.

[0176] Based on the same inventive concept, the disclosure further provides an electronic device comprising:

[0177] a memory having a computer program stored thereon;

[0178] a processor configured to execute the computer program in the memory to implement the steps of the image data processing method provided in any of the embodiments of the disclosure.

[0179] Based on the same inventive concept, the disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the image data processing method provided by any of the embodiments of the disclosure.

[0180] Figure 5 is a block diagram of an electronic device 700 according to an exemplary embodiment. As shown, the electronic device 700 can include a processor 701, a memory 702. The electronic device 700 can also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705. Figure 5

[0181] ​The processor 701 is configured to control overall operations of the electronic device 700 to complete all or part of the steps of the image data processing method described above. The memory 702 is configured to store various types of data to support operations of the electronic device 700, which can include, for example, instructions for operating any application or method on the electronic device 700, and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk. The multimedia component 703 can include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 702 or transmitted through the communication component 705. The audio component further includes at least one speaker configured to output audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 705 is configured to perform wired or wireless communication between the electronic device 700 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so the corresponding communication component 705 can include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0182] In an exemplary embodiment, the electronic device 700 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-mentioned image data processing method.

[0183] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-mentioned image data processing method. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, which can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned image data processing method.

[0184] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a processor, and when the computer program is executed by the processor, the steps of the above-mentioned image data processing method are implemented.

[0185] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above-described embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0186] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, various possible combinations are not described again in the present disclosure.

[0187] Furthermore, any combination of the various different embodiments of the present disclosure can also be made, as long as it does not deviate from the idea of the present disclosure, and it should also be considered as disclosed by the present disclosure.

Claims

1. An image data processing method, characterized in that, For a graphics processor, the method includes: The original image data of the target image is acquired and stored in the storage buffer of the graphics processor. The original image data is in a first format, which is a format in which the multi-bit data of pixels are arranged in layers according to different bit planes. The graphics processor's fragment shader retrieves the original image data of the target image from the storage buffer. Based on the original image data, an independent thread is allocated to each pixel of the target image, so that multiple threads can perform data processing for a single pixel in parallel, and each thread outputs pixel transformation data for a single pixel. The data processing for a single pixel includes: retrieving multiple valid bit data of the pixel from the storage buffer using a mask, and combining the multiple valid bit data of the pixel into pixel transformation data for the pixel. Based on the pixel conversion data of each pixel in the target image, target image data conforming to a second format is obtained, which is different from the first format.

2. The method according to claim 1, characterized in that, The original image data corresponds to multiple data layers, and each data layer stores the bit data of all pixels in the target image in a specified bit plane; The step of obtaining multiple valid bit data of the pixel from the storage buffer using a mask includes: Based on the target index of the pixel in the storage buffer, multiple data layers corresponding to the pixel are determined as the target data layers; Generate a binary bitmask based on the target index; Based on the binary bitmask, a logical AND operation is performed with the data of each of the target data layers to obtain multiple valid bit data of the pixel.

3. The method according to claim 2, characterized in that, The step of generating a binary bitmask based on the target index includes: Based on the target index, determine the bit offset of the pixel in the storage buffer; The binary bitmask is generated based on the bit offset.

4. The method according to claim 2, characterized in that, The second format corresponds to a byte order, which is used to characterize the arrangement order of color channels, wherein one byte corresponds to multiple bits of data for one color channel; The step of combining multiple valid bit data of the pixel into pixel conversion data includes: According to the arrangement order of each target data layer in the original image data, the multiple valid bit data are combined into initial stitching data; In response to determining that the byte order of the initial spliced ​​data is inconsistent with the byte order of the second format, the byte order of the initial spliced ​​data is adjusted to obtain pixel conversion data that conforms to the byte order of the second format; In response to determining that the byte order corresponding to the initial spliced ​​data is consistent with the byte order corresponding to the second format, the initial spliced ​​data is determined as the pixel conversion data.

5. The method according to claim 1, characterized in that, The storage buffer includes a shader storage buffer object; The shader storage buffer object is generated in the following way: Obtain configuration information for the shader storage buffer object, the configuration information including memory layout rules; Determine the attribute information of the shader storage buffer object, wherein the attribute information includes at least the memory layout rules; Based on the attribute information, the shader storage buffer object is created in the graphics processor.

6. The method according to claim 5, characterized in that, The attribute information also includes the lifecycle type of the shader storage buffer object and / or the data storage capacity; the lifecycle type includes temporary and persistent. If the attribute information includes the lifecycle type, determining the attribute information of the shader storage buffer object includes: Obtain the frequency at which the graphics processor processes the image; If the frequency is less than or equal to a preset threshold, the lifecycle type is determined to be the temporary type; If the frequency is greater than the preset threshold, the lifecycle type is determined to be the continuous type; If the attribute information includes the data storage capacity, determining the attribute information of the shader storage buffer object includes: The data storage capacity is determined based on the image width, image height, and bit depth of the target image.

7. The method according to claim 1, characterized in that, In the second format, the bit data of a single pixel is stored continuously, and the data of adjacent pixels are arranged continuously. The step of obtaining target image data conforming to the second format based on pixel conversion data of each pixel in the target image includes: According to the coordinates of the pixels in the target image, the pixel conversion data output by each thread is stored sequentially to obtain target image data that conforms to the second format.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: Based on the target image data, determine the color output value corresponding to each pixel in the target image; The color output value is input into the framebuffer object to render and output the framebuffer object.

9. An image data processing apparatus, characterized in that, For a graphics processor, the device includes: The first acquisition module is used to acquire the original image data of the target image and store the original image data in the storage buffer of the graphics processor. The original image data is in a first format, which is a format in which the multi-bit data of pixels are arranged in layers according to different bit planes. The processing module is configured to obtain the original image data of the target image from the storage buffer through the fragment shader of the graphics processor, and based on the original image data, allocate an independent thread for each pixel of the target image so that multiple threads can perform data processing for a single pixel in parallel, so that each thread outputs pixel transformation data for a single pixel; wherein, the data processing for a single pixel includes: obtaining multiple valid bit data of the pixel from the storage buffer using a mask, and combining the multiple valid bit data of the pixel into pixel transformation data for the pixel; The generation module is used to obtain target image data conforming to a second format based on the pixel conversion data of each pixel in the target image, wherein the second format is different from the first format.

10. A graphics processor, characterized in that, The graphics processor is used to perform the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-8.

12. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-8.

13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-8.

Images