Image processing methods, apparatus, devices and storage media

Abstract

This disclosure provides an image processing method, apparatus, device, and storage medium, relating to the field of image processing technology. It solves the problem in existing technologies where GPUs render pixel data in YUV storage format to ARGB format pixel data with delayed image updates, impacting user experience. The specific technical solution is as follows: the CPU acquires image data to be processed, which is YUV data stored in a first buffer; the Y component of a first sub-region of the first buffer is stored in a first sub-region of a second buffer; the U component of a second sub-region of the first buffer and the V component of a third sub-region of the first buffer are interleaved and stored in a second sub-region of the second buffer to obtain target image data; the address information of the second buffer is sent to a display device so that the display device can retrieve and display the target image data from the second buffer. This invention aims to accelerate image updates and improve user experience.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to image processing methods, apparatus, devices, and storage media. Background Technology

[0002] Currently, the pixel data of the image output after decoding is stored in YUV444 Planar format. However, terminal devices, such as video output processors (VOPs) of zero-terminal image display devices, do not support displaying YUV444 Planar format pixel data. They typically support displaying NV12 (YUV420SP), NV16 (YUV422SP), NV24 (YUV444SP), or ARGB8888 format pixel data. To display the decoded image on a zero-terminal device, the pixel data stored in YUV444 Planar format needs to be converted to any one of NV12 (YUV420SP), NV16 (YUV422SP), NV24 (YUV444SP), or ARGB8888.

[0003] In the prior art, the graphics processing unit (GPU) in the zero terminal renders the pixel data in YUV444 Planar storage format output by the decoder into pixel data in ARGB8888 format, and then uses the ARGB8888 format pixel data as input to the VOP for display.

[0004] However, since the rendering of YUV to ARGB must be performed in the GPU's dedicated buffer (render buffer), the GPU needs to load the YUV444 Planar pixel data from the decoding output buffer into the render buffer before rendering the YUV444 Planar pixel data. In embedded zero-terminal systems, the time it takes for the GPU to load 1080P YUV444 Planar pixel data into the render buffer can be more than 12 milliseconds. Adding the 2 to 3 millisecond rendering delay, this can result in a total latency of more than 14 milliseconds. This causes a lag in screen updates after mouse input, negatively impacting the user experience. Summary of the Invention

[0005] This disclosure provides an image processing method, apparatus, device, and storage medium that solves the problem of delayed image updates and poor user experience when the GPU renders pixel data in YUV444 Planar storage format into ARGB8888 format pixel data, and then uses the ARGB8888 format pixel data as input to a VOP for display. The technical solution is as follows:

[0006] According to a first aspect of the present disclosure, an image processing method is provided, applied to a central processing unit (CPU), the method comprising:

[0007] Acquire image data to be processed, wherein the image data to be processed is YUV data stored in a first buffer, and the first sub-region, the second sub-region, and the third sub-region of the first buffer respectively store the Y component, U component, and V component of the YUV data;

[0008] Store the Y component in the first sub-region of the first buffer into the first sub-region of the second buffer;

[0009] The U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer are interleaved and stored in the second sub-region of the second buffer to obtain the target image data;

[0010] The address information of the second buffer is sent to the display device so that the display device can obtain the target image data from the second buffer and display the target image data.

[0011] The image processing method provided in this embodiment of the present disclosure does not require loading the YUV format image data to be processed into the rendering buffer, thereby avoiding the latency of loading the YUV format image data to be processed into the rendering buffer, resulting in faster screen updates and a better user experience.

[0012] In one embodiment, the step of interleaving the U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer in the second sub-region of the second buffer includes:

[0013] M U components are obtained from the second sub-region of the first buffer, and the M U components are stored in the first address space of the register;

[0014] M V components are obtained from the third sub-region of the first buffer and stored in the second address space of the register, wherein the second address space is adjacent to the first address space;

[0015] The U and V components in the register are read intermittently according to a preset index table, and the read U and V components are stored in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U and V components in the register according to the data extraction address offset in the preset index table.

[0016] By cross-reading the U and V components in the register according to a preset index table and storing the read U and V components in the second sub-region of the second buffer, the YUV format image data to be processed can be converted into the NV format target image data.

[0017] In one embodiment, the preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offsets of each U component and each V component are arranged sequentially and interleaved in the preset index table. The step of interleaving the U component and V component in the register according to the preset index table includes:

[0018] According to the preset index table, each U component in the first address space and each V component in the second address space are read sequentially and crosswise.

[0019] By sequentially and interleaving each U component in the first address space and each V component in the second address space according to a preset index table, it is possible to interleave and store the UV components in the second sub-region of the second buffer.

[0020] In one embodiment, the preset index table includes a first preset index table and a second preset index table. The first preset index table includes the address offsets of the first N U components in the first address space and the address offsets of the first N V components in the second address space. The address offsets of the first N U components and the address offsets of the first N V components are arranged sequentially and alternately in the first preset index table. The second preset index table includes the address offsets of the last P U components in the first address space and the address offsets of the last P V components in the second address space. The address offsets of the last P U components and the address offsets of the last P V components are arranged sequentially and alternately in the second preset index table. Wherein, N + P = M.

[0021] In one embodiment, before acquiring the image data to be processed, the method further includes:

[0022] Decode the encoded image data to generate the image data to be processed;

[0023] The image data to be processed is stored in the first buffer.

[0024] By decoding the encoded image data, it is possible to generate YUV format image data to be processed.

[0025] A second aspect of this disclosure provides an image processing apparatus applied to a central processing unit (CPU), the apparatus comprising:

[0026] The image data acquisition module is used to acquire image data to be processed, wherein the image data to be processed is YUV data stored in a first buffer, and the first sub-region, the second sub-region and the third sub-region of the first buffer respectively store the Y component, U component and V component of the YUV data;

[0027] The Y component storage module is used to store the Y component in the first sub-region of the first buffer to the first sub-region of the second buffer.

[0028] The cross-storage module is used to cross-store the U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer in the second sub-region of the second buffer to obtain target image data;

[0029] The address information sending module is used to send the address information of the second buffer to the display device, so that the display device can obtain the target image data from the second buffer and display the target image data.

[0030] In one embodiment, the cross-storage module includes:

[0031] The U-component acquisition module is used to acquire M U-components from the second sub-region of the first buffer and store the M U-components into the first address space of the register;

[0032] The V component acquisition module is used to acquire M V components from the third sub-region of the first buffer and store the M V components in the second address space of the register, wherein the second address space is adjacent to the first address space;

[0033] The cross-read module is used to cross-read the U component and V component in the register according to a preset index table, and store the read U component and V component in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U component and V component in the register according to the data extraction address offset in the preset index table.

[0034] In one embodiment, the preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offsets of each U component and each V component are arranged sequentially and interleaved in the preset index table. The interleaving reading module is specifically used for:

[0035] According to the preset index table, each U component in the first address space and each V component in the second address space are read sequentially and crosswise.

[0036] In one embodiment, the preset index table includes a first preset index table and a second preset index table. The first preset index table includes the address offsets of the first N U components in the first address space and the address offsets of the first N V components in the second address space. The address offsets of the first N U components and the address offsets of the first N V components are arranged sequentially and alternately in the first preset index table. The second preset index table includes the address offsets of the last P U components in the first address space and the address offsets of the last P V components in the second address space. The address offsets of the last P U components and the address offsets of the last P V components are arranged sequentially and alternately in the second preset index table. Wherein, N + P = M.

[0037] In one embodiment, the apparatus further includes:

[0038] The image data to be processed generation module is used to decode the encoded image data and generate the image data to be processed;

[0039] The image data storage module is used to store the image data to be processed in the first buffer.

[0040] A third aspect of this disclosure provides an image processing apparatus, the image processing apparatus including a processor and a memory, the memory storing at least one computer instruction, the instruction being loaded and executed by the processor to perform the steps performed in the image processing method of any of the first aspects.

[0041] A fourth aspect of this disclosure provides a computer-readable storage medium, characterized in that the storage medium stores at least one computer instruction, which is loaded and executed by a processor to perform the steps performed in the image processing method of any one of the first aspects.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0044] Figure 1 This is a flowchart of an image processing method provided in an embodiment of the present disclosure. Figure 1 ;

[0045] Figure 2 This is a schematic diagram of YUV format pixel data provided in an embodiment of this disclosure;

[0046] Figure 3 This is a schematic diagram of NV format pixel data provided in an embodiment of this disclosure;

[0047] Figure 4 This is a schematic diagram illustrating the conversion of YUV format pixel data into NV format pixel data according to an embodiment of this disclosure;

[0048] Figure 5 This is a flowchart of an image processing method provided in an embodiment of the present disclosure. Figure 2 ;

[0049] Figure 6 This is the structure of an image processing apparatus provided in an embodiment of the present disclosure. Figure 1 ;

[0050] Figure 7 This is the structure of an image processing apparatus provided in an embodiment of the present disclosure. Figure 2 ;

[0051] Figure 8 This is a structural diagram of an image processing device provided in an embodiment of this disclosure. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0053] This disclosure provides an image processing method applied to a central processing unit (CPU). For example... Figure 1 As shown, the data transmission method includes the following steps:

[0054] S101. Obtain image data to be processed. The image data to be processed is YUV data stored in a first buffer. The first sub-region, the second sub-region, and the third sub-region of the first buffer respectively store the Y component, U component, and V component of the YUV data.

[0055] For example, the image data to be processed is encoded image data. The CPU decodes the encoded image data to generate the image data to be processed; then, the image data to be processed is stored in a first buffer. The image data to be processed is pixel data in YUV444 Planar format, and the Y, U, and V components of the YUV444 Planar format pixel data are stored in a first sub-region, a second sub-region, and a third sub-region of the first buffer, respectively.

[0056] S102, Store the Y component in the first sub-region of the first buffer into the first sub-region of the second buffer.

[0057] In this embodiment, the CPU can obtain the image data to be processed from the first buffer and store the Y component in the first sub-region to the first sub-region of the second buffer. Here, the image to be processed is described as 1920*1080 YUV444 Planar format pixel data. 1920*1080 YUV444 Planar format pixel data includes 1920*1080 Y components, U components, and V components. The CPU obtains 1920*1080 Y components from the first sub-region of the first buffer and stores them in the first sub-region of the second buffer. Each U / V component is one byte.

[0058] S103. The U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer are interleaved and stored in the second sub-region of the second buffer to obtain the target image data.

[0059] For example, the CPU executes a first Vector Load (VLD) instruction, which first retrieves M U components from the 1920*1080 U components stored in the second sub-region of the first buffer, and stores these M U components in the first address space of the NEON register. M can be 8, 16, or any other multiple of 8. Preferably, in this embodiment, M is 8.

[0060] Similarly, the CPU executes a second Vector Load (VLD) instruction, first retrieving 8 V components from the 1920*1080 V components stored in the third sub-region of the first buffer, and storing these 8 V components in the second address space of the NEON register, which is adjacent to the first address space. For example, the NEON register is a 128-bit register. After retrieving the 8 U components, the CPU stores them in the first 64 bits of the NEON register, i.e., the first address space of the NEON register. After retrieving the 8 V components, the CPU stores them in the last 64 bits of the NEON register, i.e., the second address space of the NEON register.

[0061] After storing the 8 U components and 8 V components into the NEON register, the U components and V components in the NEON register are read interleaved according to a preset index table, and the read U components and V components are stored in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U components and V components in the NEON register according to the data extraction address offset in the preset index table.

[0062] The following explains how the CPU reads the U and V components of the NEON register in a cross-referencing manner according to the preset index table.

[0063] For example, the preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offset of each U component and the address offset of each V component are arranged sequentially and interleaved in the preset index table. The CPU reads each U component in the first address space and each V component in the second address space sequentially and interleaved according to the preset index table, and then stores each U component and each V component read in the second address space into the second sub-region of the second buffer.

[0064] For example, in this embodiment, the first 64 bits of the NEON register store eight U components, namely U0, U1, U2, U3, U4, U5, U6, and U7, with address offsets of 0, 1, 2, 3, 4, 5, 6, and 7, respectively. The last 64 bits of the NEON register store eight V components, namely V0, V1, V2, V3, V4, V5, V6, and V7, with address offsets of 8, 9, 10, 11, 12, 13, 14, and 15, respectively. Therefore, the preset index table is 0, 8, 1, 9, 2, 10, 3, 11, 4, 12, 5, 13, 6, 14, 7, and 15. The CPU sequentially and interleavedly reads each U component in the first address space and each V component in the second address space according to the preset index table, namely U0, V0, U1, V1, U2, V2, U3, V3, U4, V4, U5, V5, U6, V6, U7, V7. Then, it stores the read U0, V0, U1, V1, U2, V2, U3, V3, U4, V4, U5, V5, U6, V6, U7, V7 into the second sub-region of the second buffer. The U and V components stored in the second sub-region of the second buffer are interleaved.

[0065] Further, after storing U0 to V7 in the second sub-region of the second buffer, the offset address of the U component in the second sub-region of the first buffer is increased by 8 bytes, and the offset address of the V component in the third sub-region of the first buffer is increased by 8 bytes. The CPU then executes the first VLD instruction to obtain 8 U components U8 to U15 from the second sub-region of the first buffer and stores them in the first address space of the NEON register. The CPU then executes the second VLD instruction to obtain 8 V components V8 to V15 from the third sub-region of the first buffer and stores them in the second address space of the NEON register. Then, according to a preset index table, the CPU cross-reads the U8 to U15 and V8 to V15 components in the NEON register, increases the offset address of the UV components in the second sub-region of the second buffer by 16 bytes, and stores the cross-read U8 to U15 and V8 to V15 components in the second region of the second buffer. The U8 to V15 components are stored adjacent to the U0 to V7 components in the second sub-region.

[0066] Further, the above process is repeated until all 1920*1080 U components from the second sub-region of the first buffer and all 1920*1080 V components from the third sub-region of the first buffer are interleaved and stored in the second region of the second buffer, generating target image data, which is NV24 format pixel data. NV24 format pixel data stores the Y component separately, while the U and V components are interleaved. Using the method of this embodiment, image data to be processed can be converted into NV24 format pixel data.

[0067] In one embodiment, the preset index table may include a first preset index table and a second preset index table. The first preset index table includes the address offsets of the first N U components in the first address space and the address offsets of the first N V components in the second address space. The address offsets of the first N U components and the address offsets of the first N V components are arranged sequentially and alternately in the first preset index table. The second preset index table includes the address offsets of the last P U components in the first address space and the address offsets of the last P V components in the second address space. The address offsets of the last P U components and the address offsets of the last P V components are arranged sequentially and alternately in the second preset index table. Wherein, N+P=M.

[0068] Here, we will use 4 for both N and P. In this embodiment, the first 64 bits of the NEON register store 8 U components, namely U0, U1, U2, U3, U4, U5, U6, and U7, with address offsets of 0, 1, 2, 3, 4, 5, 6, and 7, respectively. The last 64 bits of the NEON register store 8 V components, namely V0, V1, V2, V3, V4, V5, V6, and V7, with address offsets of 8, 9, 10, 11, 12, 13, 14, and 15, respectively. Therefore, the first preset index table is 0, 8, 1, 9, 2, 10, 3, and 11, and the second preset index table is 4, 12, 5, 13, 6, 14, 7, and 15. The CPU executes a first Vector Table (VTBL) instruction, sequentially and interleaving the first four U components in the first address space and the first four V components in the second address space according to the first preset index table, namely U0, V0, U1, V1, U2, V2, U3, V3. Then, it stores the read U0, V0, U1, V1, U2, V2, U3, V3 into the second sub-region of the second buffer. The U and V components stored in the second sub-region of the second buffer are interleaved.

[0069] Furthermore, after storing U0 to V4 in the second sub-region of the second buffer, the CPU executes the second VTBL instruction, sequentially and interleaving the last four U components in the first address space and the last four V components in the second address space according to the second preset index table, namely U4, V4, U5, V5, U6, V6, U7, V7. The offset address of the UV components in the second sub-region of the second buffer is increased by 8 bytes, and then the read U4, V4, U5, V5, U6, V6, U7, V7 are stored in the second sub-region of the second buffer. The U components and V components stored in the second sub-region of the second buffer are interleaved, and U4 to V7 are stored adjacent to U0 to V3 in the second sub-region of the second buffer.

[0070] Furthermore, after storing U0 to V7 in the second sub-region of the second buffer, the offset address of the U component in the second sub-region of the first buffer is increased by 8 bytes, and the offset address of the V component in the third sub-region of the first buffer is increased by 8 bytes. The CPU then executes the first VLD instruction to obtain 8 U components U8 to U15 from the second sub-region of the first buffer and stores the 8 U components U8 to U15 in the first address space of the NEON register. The CPU then executes the second VLD instruction to retrieve eight V components V8 to V15 from the third sub-region of the first buffer, and increments the offset address of the UV components in the second buffer by eight bytes. The CPU then stores the eight V components V8 to V15 into the second address space of the NEON register. Next, the CPU executes the first VTBL instruction to cross-read the U8 to U11 components and V8 to V11 components in the NEON register according to the first preset index table, and increments the offset address of the UV components in the second sub-region of the second buffer by eight bytes. The CPU then stores the cross-read U8 to U11 components and V8 to V11 components into the second region of the second buffer. The U8 to V11 components are adjacent to the U4 to V7 components in the second sub-region of the second buffer.

[0071] After the CPU stores U8 to V11 into the second sub-region of the second buffer, it executes the second VTBL instruction to cross-read the U12 to V15 components and V12 to V15 components in the NEON register according to the second preset index table, increases the offset address of the UV components in the second buffer by 8 bytes, and then stores the cross-read U12 to U15 components and V12 to V15 components into the second region of the second buffer. The U12 to V15 components are adjacent to the U8 to V11 components in the second sub-region.

[0072] Further, the above process is repeated until all 1920*1080 U components from the second sub-region of the first buffer and 1920*1080 V components from the third sub-region of the first buffer are interleaved and stored in the second region of the second buffer, generating target image data, which is pixel data in NV24 format. NV24 format pixel data stores the Y component separately, while the U and V components are interleaved.

[0073] S104. The address information of the second buffer is sent to the display device so that the display device can obtain the target image data from the second buffer and display the target image data.

[0074] For example, the display device can be a VOP. The CPU interleave and stores all 1920*1080 U components in the second sub-region of the first buffer and 1920*1080 V components in the third sub-region of the first buffer into the second region of the second buffer. After generating NV24 format target image data, the CPU sends the address information of the second buffer to the VOP so that the VOP can obtain the NV24 format target image data from the second buffer and display the NV24 format target image data.

[0075] The image processing method provided in this embodiment can acquire image data to be processed, which is YUV data stored in a first buffer. The first sub-region, second sub-region, and third sub-region of the first buffer respectively store the Y component, U component, and V component of the YUV data. The Y component in the first sub-region of the first buffer is stored in the first sub-region of the second buffer. The U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer are interleaved and stored in the second sub-region of the second buffer to obtain target image data. The address information of the second buffer is sent to the display device so that the display device can obtain the target image data from the second buffer and display the target image data. The CPU can convert the YUV format image data to be processed into NV format target image data, and the image display device can obtain the NV format target image data from the second buffer and display the target image data. There is no need to load the YUV format image data to be processed into the rendering buffer, thereby avoiding the latency of loading the YUV444Planar format image data to be processed into the rendering buffer, resulting in faster screen updates and a better user experience.

[0076] The following is combined with Figures 2 to 5 The image processing method provided in the embodiments of this disclosure will be further described in detail.

[0077] Specifically, the YUV444 Planar storage format is as follows: Figure 2 As shown. Start0-0 to start0-15 (i.e., the first sub-region of the first buffer) represents Plane 0 (16 bytes) corresponding to the Y component of the YUV444 Planar. Start1-0 to start1-15 (i.e., the second sub-region of the first buffer) represents Plane 1 (16 bytes) corresponding to the U component of the YUV444 Planar. Start2-0 to start2-15 (i.e., the third sub-region of the first buffer) represents Plane 2 (16 bytes) corresponding to the V component of the YUV444 Planar.

[0078] NV24 storage format such as Figure 3 As shown. Start0 to start15 (i.e., the first sub-region of the second buffer) represents Plane0 (16 bytes) corresponding to the Y component of NV24. Start16 to start47 (i.e., the second sub-region of the second buffer) represents Plane1 (32 bytes) corresponding to the UV component of NV24.

[0079] The method for converting pixel data from YUV444 Planar storage format to NV24 storage format according to the present invention is described below:

[0080] In this invention, the conversion from YUV444 Planar storage format to NV24 storage format involves directly copying the Y component from the YUV444 Planar to the Y component plane in the NV24 without conversion. However, for the UV components, ARM NEON instructions are used to cross-move the UV components from their respective planes in the YUV444 Planar to the UV component planes in the NV24.

[0081] Currently, the U and V components can be read byte by byte from the U component plane and V component plane of the YUV444 Planar (the instruction memory access unit is 1 byte), and the bytes of the UV components are interleaved and stored in the UV component plane of the NV24. This conversion rate is very slow (only one byte can be processed in one instruction cycle).

[0082] To address the conversion process of interleaving the UV components in a YUV444 Planar from their respective Planes to the UV component Planes in an NV24, this invention proposes an optimized conversion method. This method can optimize the process of converting the two Planes of U and V in the YUV444 Planar format into a second UV-merged Plane in NV24, processing an 8-byte vector at a time (the instruction memory access unit is 8 bytes), and using fewer instruction cycles than the aforementioned methods, thus improving conversion efficiency.

[0083] The optimized conversion method proposed in this invention uses two NEON instructions: VLD and VTBL.

[0084] The VLD instruction loads an 8-byte vector at a time. During each processing iteration, VLD can be used twice consecutively to load 8 bytes from the U component plane and 8 bytes from the V component plane of the YUV444 Planar, respectively.

[0085] The VTBL instruction is a table lookup instruction. In each processing iteration, the VTBL instruction can be used twice consecutively, with different index tables used for each instruction. Using these index tables, the consecutively stored U and V components obtained after executing the VLD instruction are interleaved and stored in the UV component plane.

[0086] refer to Figure 4 , Figure 4 A schematic diagram of the conversion process, such as Figure 4 The specific conversion process is explained below:

[0087] S501. Completely copy Plane 0 (Y component) from YUV444 Planar to Plane 0 (Y component) from NV24.

[0088] S502. The first VLD instruction is executed, loading U0-U7 from Plane 1 (U component) in YUV444 Planar into the first 64 bits (i.e., the first address space) of the NEON register (U0-U7).

[0089] Each U / Y component is one byte, and one byte occupies 8 bits.

[0090] S503. The second VLD instruction is executed, loading V0-V7 from Plane2 (V component) in YUV444 Planar into the last 64 bits (i.e., the second address space) of the NEON register (V0-V7).

[0091] In this step, VLD loads another 8 bytes into the NEON registers (V0-V7). The NEON registers are 128 bits in total and can store 16 bytes of data.

[0092] It should be noted that the processing in this step is completed in the NEON register and does not involve memory access operations. That is, step 2 fills the first 64 bits of the 128-bit register with an 8-byte vector U, and step 3 fills the last 64 bits of the 128-bit register with an 8-byte vector V. By sequentially connecting U0-U7 loaded in step 2 with V0-V7 loaded in step 3, a 16-byte vector can be formed, which provides input for the next table lookup.

[0093] S504. The first VTBL instruction is executed. Based on the 8-byte vector "0819210311" in index table 0, bytes with offsets of 0, 8, 1, 9, 2, 10, 3, and 11 are extracted from the NEON register and stored sequentially into NV24 Plane1 (UV component) to obtain the interleaved U0 V0 U1 V1 U2 V2 U3 V3.

[0094] In this step, bytes in the register are read and stored according to index table 0. The numbers in the index table correspond to the offset byte number of the NEON register.

[0095] For example, if "0" in index table 0 corresponds to U0 at offset 0 bytes in the 16-byte vector of the NEON register, then U0 can be written to the position at offset 0 bytes in Plane1 (UV component) of NV24; if "8" in index table 0 corresponds to V0 at offset 8 bytes in the 16-byte vector of the NEON register, then V0 can be written to the position at offset 1 byte in Plane1 (UV component) of NV24; and if "1" in the index table corresponds to U1 at offset 1 byte in the 16-byte vector of the NEON register, then U1 can be written to the position at offset 2 bytes in Plane1 (UV component) of NV24.

[0096] As can be seen, based on the 8-byte vector in index table 0, the 16 bytes in the NEON register can be written to positions from offset 0 to offset 7 in Plane1 (UV component) of NV24. The bytes in the NEON register are interleaved when written to Plane1 (UV component) of NV24.

[0097] S505, the second execution of the VTBL instruction (i.e. the first VTBL instruction) extracts the byte with offset 412513614715 from the NEON register according to the 8-byte vector "412513614715" in index table 1, and stores the above byte into Plane1 (UV component) of NV24 in sequence, to obtain the interleaved U4 V4 U5 V5 U6 V6 U7 V7.

[0098] In this step, similar to step 4, the remaining 8 bytes in the NEON register can be interleaved and filled into the Plane1 (UV component) of NV24 at an offset of 8 bytes to 15 bytes, according to the 8 vectors in index table 1.

[0099] S506. Add 8 bytes to the read offset addresses of plane1 (U component) and plane2 (V component) in the YUV444 Planar.

[0100] The write offset address of Plane1 (UV component) in S507 and NV24 is increased by 8 bytes.

[0101] S502 to S507 are executed sequentially until all data in YUV444 Planar Plane1 (U component) and Plane2 (V component) are written into NV24 Plane1 (UV component), and the conversion is complete.

[0102] Understandably, each execution from S502 to S505 completes the conversion of 8 bytes of the U component and 8 bytes of the V component in YUV444. In S506 and S507, the offset address needs to be adjusted in order to continue reading and converting the subsequent U and V components.

[0103] Based on the above Figures 1 to 5 The image processing methods described in the corresponding embodiments are described below as device embodiments of this disclosure, which can be used to execute the method embodiments of this disclosure.

[0104] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure, which is applied to a central processing unit (CPU). Figure 6 As shown, the device 60 includes:

[0105] The image data acquisition module 601 is used to acquire image data to be processed, wherein the image data to be processed is YUV data stored in a first buffer, and the first sub-region, the second sub-region and the third sub-region of the first buffer respectively store the Y component, U component and V component of the YUV data.

[0106] Y component storage module 602 is used to store the Y component in the first sub-region of the first buffer to the first sub-region of the second buffer;

[0107] The cross-storage module 603 is used to cross-store the U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer in the second sub-region of the second buffer to obtain target image data.

[0108] Address information sending module 604 is used to send the address information of the second buffer to the display device, so that the display device can obtain the target image data from the second buffer and display the target image data.

[0109] In one embodiment, the cross-storage module 603 is used for:

[0110] The U-component acquisition module is used to acquire M U-components from the second sub-region of the first buffer and store the M U-components into the first address space of the register;

[0111] The V component acquisition module is used to acquire M V components from the third sub-region of the first buffer and store the M V components in the second address space of the register, wherein the second address space is adjacent to the first address space;

[0112] The cross-read module is used to cross-read the U component and V component in the register according to a preset index table, and store the read U component and V component in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U component and V component in the register according to the data extraction address offset in the preset index table.

[0113] In one embodiment, the preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offsets of each U component and each V component are sequentially and interleaved in the preset index table. The interleaving reading module 603 is specifically used for:

[0114] According to the preset index table, each U component in the first address space and each V component in the second address space are read sequentially and crosswise.

[0115] In one embodiment, the preset index table includes a first preset index table and a second preset index table. The first preset index table includes the address offsets of the first N U components in the first address space and the address offsets of the first N V components in the second address space. The address offsets of the first N U components and the address offsets of the first N V components are arranged sequentially and alternately in the first preset index table. The second preset index table includes the address offsets of the last P U components in the first address space and the address offsets of the last P V components in the second address space. The address offsets of the last P U components and the address offsets of the last P V components are arranged sequentially and alternately in the second preset index table. Wherein, N + P = M.

[0116] In one embodiment, such as Figure 7 As shown, the device 60 also includes:

[0117] The image data generation module 605 is used to decode the encoded image data to generate the image data to be processed;

[0118] The image data storage module 606 is used to store the image data to be processed in the first buffer.

[0119] The software application configuration device provided in this disclosure can be implemented and its technical effects can be found in the above-described embodiments. Figures 1 to 5 Examples are not described in detail here.

[0120] Figure 8 This is a schematic diagram of the structure of the image processing device provided in the embodiments of this disclosure, such as... Figure 8 As shown, the image processing device 80 includes:

[0121] A processor 801 and a memory 802, wherein the memory 801 stores at least one computer instruction, which is loaded and executed by the processor 801 to implement [the desired function]. Figures 1 to 5 The steps performed in the image processing method described in the corresponding embodiment.

[0122] Based on the above Figures 1 to 5 In addition to the image processing method described in the corresponding embodiments, this disclosure also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage device. This storage medium stores computer instructions for executing the above-described methods. Figures 1 to 5 The image processing methods described in the corresponding embodiments will not be repeated here.

[0123] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0124] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

Claims

1. An image processing method, characterized in that, Applied to a central processing unit (CPU), the method includes: Acquire image data to be processed, wherein the image data to be processed is YUV data stored in a first buffer, and the first sub-region, the second sub-region, and the third sub-region of the first buffer respectively store the Y component, U component, and V component of the YUV data; Store the Y component in the first sub-region of the first buffer into the first sub-region of the second buffer; The U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer are interleaved and stored in the second sub-region of the second buffer to obtain the target image data; The address information of the second buffer is sent to the display device so that the display device can obtain the target image data from the second buffer and display the target image data; The step of interleaving the U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer in the second sub-region includes: M U components are obtained from the second sub-region of the first buffer, and the M U components are stored in the first address space of the register; M V components are obtained from the third sub-region of the first buffer and stored in the second address space of the register, wherein the second address space is adjacent to the first address space; The U and V components in the register are read intermittently according to a preset index table, and the read U and V components are stored in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U and V components in the register according to the data extraction address offset in the preset index table.

2. The method according to claim 1, characterized in that, The preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offsets of each U component and each V component are arranged sequentially and alternately in the preset index table. The step of reading the U component and V component in the register alternately according to the preset index table includes: According to the preset index table, each U component in the first address space and each V component in the second address space are read sequentially and crosswise.

3. The method according to claim 2, characterized in that, The preset index table includes a first preset index table and a second preset index table. The first preset index table includes the address offsets of the first N U components in the first address space and the address offsets of the first N V components in the second address space. The address offsets of the first N U components and the address offsets of the first N V components are arranged sequentially and alternately in the first preset index table. The second preset index table includes the address offsets of the last P U components in the first address space and the address offsets of the last P V components in the second address space. The address offsets of the last P U components and the address offsets of the last P V components are arranged sequentially and alternately in the second preset index table. Wherein, N + P = M.

4. The method according to any one of claims 1 to 3, characterized in that, Before acquiring the image data to be processed, the method further includes: Decode the encoded image data to generate the image data to be processed; The image data to be processed is stored in the first buffer.

5. An image processing apparatus, characterized in that, The device, applied to a central processing unit (CPU), includes: The image data acquisition module is used to acquire image data to be processed, wherein the image data to be processed is YUV data stored in a first buffer, and the first sub-region, the second sub-region and the third sub-region of the first buffer respectively store the Y component, U component and V component of the YUV data; The Y component storage module is used to store the Y component in the first sub-region of the first buffer to the first sub-region of the second buffer. The cross-storage module is used to cross-store the U component in the second sub-region of the first buffer and the V component in the third sub-region of the first buffer in the second sub-region of the second buffer to obtain target image data; The address information sending module is used to send the address information of the second buffer to the display device, so that the display device can obtain the target image data from the second buffer and display the target image data; The cross-storage module includes: The U-component acquisition module is used to acquire M U-components from the second sub-region of the first buffer and store the M U-components into the first address space of the register; The V component acquisition module is used to acquire M V components from the third sub-region of the first buffer and store the M V components in the second address space of the register, wherein the second address space is adjacent to the first address space; The cross-read module is used to cross-read the U component and V component in the register according to a preset index table, and store the read U component and V component in the second sub-region of the second buffer. The preset index table is used to indicate the reading of the U component and V component in the register according to the data extraction address offset in the preset index table.

6. The apparatus according to claim 5, characterized in that, The preset index table includes the address offset of each U component in the first address space and the address offset of each V component in the second address space. The address offsets of each U component and each V component are arranged sequentially and interleaved in the preset index table. The interleaving reading module is specifically used for: According to the preset index table, each U component in the first address space and each V component in the second address space are read sequentially and crosswise.

7. An image processing device, characterized in that, The image processing device includes a processor and a memory, the memory storing at least one computer instruction, which is loaded and executed by the processor to perform the steps of the image processing method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer instruction, which is loaded and executed by a processor to perform the steps in the image processing method according to any one of claims 1 to 4.

Images