An image data optimization device and method

Abstract

The present application discloses an image data optimization device and method, which sends the current frame of image data to be decoded to the VPU, enabling the VPU to decode the current frame of image data to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame. The preset decoding optimization method is determined through the VPU hardware decoding function. When it is necessary to process the next frame of image data to be decoded or an interrupt message sent by the VPU is received, the decoded image of the current frame is obtained and output through a preset acquisition method, and the preset acquisition method is used to reduce the load of the host processor. Based on the above solution, by adopting the preset decoding optimization method, that is, the hardware decoding method, the image data to be decoded is decoded to obtain the decoded image data and output. Decoding through the hardware decoding method has relatively low requirements for the computing power of the VPU, thereby improving the decoding efficiency of the image data to be decoded, and at the same time reducing the load of the host processor and avoiding video stuttering.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and more specifically, to an image data optimization device and method. Background Art

[0002] In recent years, with the emergence of new application scenarios of remote display systems, how to meet the access requirements of various lightweight clients has become an important issue in remote display. The screen content encoding technology in the remote display system Spice well solves this problem. After obtaining the screen content, encoding and transcoding it, it is transmitted to the client through the network. The client only performs the work of decoding and displaying, so the computing pressure on the client is effectively reduced, and the platform independence of the client is enhanced, thus meeting the needs of lightweight clients.

[0003] In the existing process of remote display through screen content encoding technology, when a video or image reaches the client, the technology, namely the Motion Joint Photographic Experts Group (MJPEG) compression technology for motion still images (or frame-by-frame), is used for software decoding. This decoding process requires a high computing power of the client. When the computing power of the client is insufficient, it will cause excessive decoding time, resulting in a decrease in the actual playback frame rate and causing video stuttering. Summary of the Invention

[0004] In view of this, this application discloses an image data optimization device and method, aiming to improve the decoding efficiency of the image data to be decoded, and at the same time reduce the load of the host processor and avoid video stuttering.

[0005] To achieve the above object, the disclosed technical solution is as follows:

[0006] The first aspect of this application discloses an image data optimization device, and the device includes:

[0007] A host processor, configured to send the current frame of image data to be decoded to a video processing unit (VPU);

[0008] The VPU is configured to decode the current frame of image data to be decoded through a preset decoding optimization method to obtain the current frame of decoded image; the preset decoding optimization method is determined through a VPU hardware decoding function;

[0009] The host processor is further configured to, when it is necessary to process the next frame of image data to be decoded or receive an interrupt message sent by the VPU, obtain the current frame of decoded image through a preset acquisition method and output it; the interrupt message is used to prompt that the image decoding operation of the current frame of image data to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor.

[0010] Preferably, the VPU is specifically configured to:

[0011] Compile the VPU hardware decoding function into a dynamic link library; call the dynamic link library to decode the current frame of image data to be decoded, and obtain the decoded image of the current frame.

[0012] Preferably, the host processor is specifically configured to:

[0013] When it is necessary to process the next frame of image data to be decoded or an interrupt message sent by the VPU is received, obtain the decoded image of the current frame from the dynamic shared cache; the decoded image of the current frame is sent by the VPU to the dynamic shared cache.

[0014] Preferably, the host processor is further specifically configured to:

[0015] Send an image processing requirement to the VPU through the Host Interface Reg register, and receive a reply message corresponding to the image processing requirement sent by the VPU forwarded by the Host Interface Reg register.

[0016] Preferably, the host processor is further specifically configured to:

[0017] Perform other data processing operations other than image data processing; the current frame of image data to be decoded is sent when the host processor performs the image data processing.

[0018] A second aspect of the present application discloses an image data optimization method, and the method includes:

[0019] Send the current frame of image data to be decoded to the video processing unit VPU, so that the VPU decodes the current frame of image data to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined through the VPU hardware decoding function.

[0020] When it is necessary to process the next frame of image data to be decoded or an interrupt message sent by the VPU is received, obtain the decoded image of the current frame through a preset acquisition method and output it; the interrupt message is used to prompt that the image decoding operation of the current frame of image data to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor itself.

[0021] Preferably, the sending the current frame of image data to be decoded to the VPU, so that the VPU decodes the current frame of image data to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame includes:

[0022] Send the image data of the current frame to be decoded to the VPU, so that the VPU compiles the VPU hardware decoding function into a dynamic link library;

[0023] Call the dynamic link library to decode the image data of the current frame to be decoded, and obtain the decoded image of the current frame.

[0024] Preferably, when it is necessary to process the image data of the next frame to be decoded or an interrupt message sent by the VPU is received, the decoded image of the current frame is obtained and output through a preset acquisition method, including:

[0025] When it is necessary to process the image data of the next frame to be decoded or an interrupt message sent by the VPU is received, obtain the decoded image of the current frame from the dynamic shared cache and output it; the decoded image of the current frame is sent by the VPU to the dynamic shared cache.

[0026] Preferably, before sending the image data of the current frame to be decoded to the video processing unit VPU, so that the VPU decodes the image data of the current frame to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame, it further includes:

[0027] Send the image processing requirement to the VPU through the Host Interface Reg register, and receive the reply information corresponding to the image processing requirement forwarded by the Host Interface Reg register from the VPU.

[0028] Preferably, after sending the image data of the current frame to be decoded to the video processing unit VPU, so that the VPU decodes the image data of the current frame to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame, it further includes:

[0029] Perform other data processing operations other than image data processing; the image data of the current frame to be decoded is sent during the execution of the image data processing.

[0030] As can be seen from the above technical solutions, the present application discloses an image data optimization device and method, which sends the image data to be decoded of the current frame to a video processing unit (VPU), enabling the VPU to decode the image data to be decoded of the current frame through a preset decoding optimization method to obtain the decoded image of the current frame. The preset decoding optimization method is determined through the VPU hardware decoding function. When it is necessary to process the image data to be decoded of the next frame or an interrupt message sent by the VPU is received, the decoded image of the current frame is obtained through a preset acquisition method and output; the interrupt message is used to indicate that the image decoding operation of the image data to be decoded of the current frame has been completed; the preset acquisition method is used to reduce the load of the host processor. Based on the above solutions, by adopting the preset decoding optimization method, that is, the hardware decoding method, the image data to be decoded is decoded to obtain the decoded image data and output. Decoding through the hardware decoding method requires a lower computing power for the VPU, thereby improving the decoding efficiency of the image data to be decoded, and at the same time reducing the load of the host processor and avoiding video stuttering. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative efforts.

[0032] Figure 1 It is an architecture diagram of a remote display system disclosed in an embodiment of the present application;

[0033] Figure 2 It is a schematic structural diagram of an image data optimization device disclosed in an embodiment of the present application;

[0034] Figure 3 It is a schematic flowchart of an image data optimization method disclosed in an embodiment of the present application;

[0035] Figure 4 It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0037] In this application, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0038] As can be seen from the background art, in the existing process of remote display through screen content encoding technology, when a video or image arrives at the client, MJPEG is used for software decoding. This decoding process requires high computing power of the client. When the computing power of the client is insufficient, it will cause excessive decoding time, resulting in a decrease in the actual playback frame rate and video stuttering.

[0039] To solve the above problems, embodiments of this application disclose an image data optimization device and method. By adopting a preset decoding optimization method, i.e., hardware decoding, the image data to be decoded is decoded to obtain the decoded image data and output, improving the decoding efficiency of the image data to be decoded, reducing the load of the host processor, and avoiding video stuttering. The specific implementation is described in the following embodiments.

[0040] First, the architecture diagram of the remote display system Spice applicable to an image data optimization device and method of this application is introduced below. Specifically, refer to Figure 1 As shown, the remote display system Spice includes a Video Processing Unit (VPU) 11, a host processor 12, an SDRAM VPU dynamic buffer 13, a Host Interface Reg register 14, a Synchronous Dynamic Random-access Memory (SDRAM) Shared buffer 15, an Application Programming Interface (API) function interface 16, and a PKUnity86 platform.

[0041] The specific data processing process among the VPU 11, the host processor 12, the SDRAM VPU dynamic buffer 13, the Host Interface Reg register 14, the SDRAM Shared dynamic shared buffer 15, the API function interface 16, and the PKUnity86 platform is as follows:

[0042] The host processor 12 sends image processing requirements to the VPU 11 through the Host Interface Reg register 14, and receives the reply information corresponding to the image processing requirements sent by the VPU 11 forwarded by the Host Interface Reg register 14.

[0043] Among them, the Host Interface Reg register 14 receives commands / responses sent by the host processor 12 through the API function interface 16.

[0044] The SDRAM VPU dynamic buffer 13 is the built-in dynamic buffer of the VPU 11, and the SDRAM VPU dynamic buffer 13 is used to cache the image data to be decoded.

[0045] The host processor 12 sends the image data of the current frame to be decoded to the VPU 11.

[0046] The host processor 12 performs other data processing operations other than image data processing; the image data of the current frame to be decoded is sent when the host processor 12 performs image data processing.

[0047] Among them, the host processor 12 only needs to put forward image processing requirements to the VPU 11, and then the host processor 12 can process other data processing operations other than image data processing.

[0048] The VPU 11 decodes the image data of the current frame to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined through the hardware decoding function of the VPU 11.

[0049] Among them, the VPU 11 compiles the VPU hardware decoding function into a dynamic link library; the dynamic link library is called to decode the image data of the current frame to be decoded to obtain the decoded image of the current frame. The decoded images can form a corresponding video.

[0050] When it is necessary to process the image data of the next frame to be decoded or an interrupt message sent by the VPU 11 is received, the host processor 12 obtains the decoded image of the current frame through a preset acquisition method and outputs it; the interrupt message is used to prompt that the image decoding operation of the image data of the current frame to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor 12.

[0051] Among them, the image data of the next frame to be decoded is the image data of the frame to be decoded at the next moment.

[0052] When the host processor 12 needs to process the next frame of image data to be decoded or receives an interrupt message sent by the VPU 11, the host processor 12 obtains the decoded image of the current frame from the SDRAM Shared dynamic shared buffer 15; the decoded image of the current frame is sent by the VPU 11 to the SDRAM Shared dynamic shared buffer 15.

[0053] The decoded images can form a video, and the host processor 12 maps the video on the PKUnity86 platform for display.

[0054] The API function interface 16 provides calls to the host application of the host processor 12 with parameters, and the API function interface 16 can encapsulate the return value after updating the parameters into the host application.

[0055] In the embodiment of the present application, by adopting a preset decoding optimization method, that is, a hardware decoding method, the image data to be decoded is decoded to obtain the decoded image data and output. Decoding by the hardware decoding method has a lower requirement for the computing power of the VPU, thereby improving the decoding efficiency of the image data to be decoded, and at the same time reducing the load of the host processor and avoiding the phenomenon of video stuttering.

[0056] Reference Figure 2 As shown, it is a schematic structural diagram of an image data optimization device disclosed in the embodiment of the present application. The image data optimization device mainly includes a host processor 201 and a VPU 202.

[0057] The host processor 201 is used to send the current frame of image data to be decoded to the VPU 202.

[0058] The VPU 202 is used to decode the current frame of image data to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined by the hardware decoding function of the VPU 202.

[0059] The host processor 201 is further used to obtain and output the decoded image of the current frame by a preset acquisition method when it is necessary to process the next frame of image data to be decoded or receives an interrupt message sent by the VPU; the interrupt message is used to prompt that the image decoding operation of the current frame of image data to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor.

[0060] Further, the VPU 202 is specifically used to compile the hardware decoding function into a dynamic link library; call the dynamic link library to decode the current frame of image data to be decoded to obtain the decoded image of the current frame.

[0061] Further, the host processor 201 is specifically configured to obtain the decoded image of the current frame from the dynamic shared cache when it is necessary to process the next frame of image data to be decoded or when an interrupt message sent by the VPU 202 is received; the decoded image of the current frame is sent by the VPU 202 to the dynamic shared cache.

[0062] Further, the host processor 201 is specifically further configured to send image processing requirements to the VPU 202 through the Host Interface Reg register, and receive the reply information corresponding to the image processing requirements sent by the VPU 202 forwarded by the Host Interface Reg register.

[0063] Further, the host processor 201 is specifically further configured to perform other data processing operations other than image data processing; the image data of the current frame to be decoded is sent when the host processor 201 performs image data processing.

[0064] In the embodiment of the present application, by adopting a preset decoding optimization method, that is, a hardware decoding method, the image data to be decoded is decoded to obtain the decoded image data and output, which improves the decoding efficiency of the image data to be decoded, reduces the load of the host processor, and avoids the phenomenon of video freezing.

[0065] Based on the above embodiment Figure 2 An image data optimization device is disclosed, and the embodiment of the present application also correspondingly discloses an image data optimization method. The image data optimization method is applied to the host processor, as Figure 3 shown, the image data optimization method mainly includes the following steps:

[0066] S301: The host processor sends the image data of the current frame to be decoded to the video processing unit VPU, so that the VPU decodes the image data of the current frame to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined by the VPU hardware decoding function.

[0067] Among them, the image data of the current frame to be decoded is the image data of the frame to be decoded at the current moment.

[0068] The VPU is a high-performance multi-format media video hardware module, and the VPU can process the decoding and encoding of videos of types such as H.264 BP / MP / HP, VC-1 SP / MP / AP, MPEG4 SP / ASP, Divx, MPEG-2 MP, RV-8 / 9 / 10, AVS, and JPEG.

[0069] An SDRAM VPU dynamic buffer is set in the VPU, and the SDRAM VPU dynamic buffer is used to cache the image frame data to be decoded.

[0070] Decoding is a process of restoring digital codes to their original represented content or converting electrical pulse signals, optical signals, radio waves, etc. into the information, data, etc. they represent. Decoding is the process by which the recipient restores the received symbols or codes into information, corresponding to the encoding process.

[0071] The VPU supports the decoding of full-screen 1920x1080 high-definition video and the encoding of 1280x720 high-definition video. At the same time, it can also process multiple video segments in parallel. Since most hardware modules in the VPU are shared to the greatest extent during the encoding and decoding of different video formats, the VPU hardware achieves extremely low power consumption and gate count.

[0072] To facilitate the use of the VPU by host users, the VPU provides a set of APIs, namely the VPU API. The VPU API includes a control API and a decoding API. The control API includes a series of API functions for basic control of the VPU, such as the initialization function of the VPU; the VPU decodes the image data of the current frame to be decoded through the decoding API.

[0073] Among them, the VPU API is the API function interface provided by the VPU externally for the host application HostApplication to call.

[0074] The host application HostApplication includes a streaming media player program, etc.

[0075] Specifically, the process of the host processor sending the image data of the current frame to be decoded to the video processing unit VPU and enabling the VPU to decode the image data of the current frame to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame is as follows:

[0076] First, the host processor sends the image data of the current frame to be decoded to the VPU, enabling the VPU to compile the VPU hardware decoding function into a dynamic link library.

[0077] Among them, the VPU hardware decoding function adopts a frame-level decoding method, which can greatly reduce the dependence of VPU API operations (control operations and decoding operations) on the host processor.

[0078] To facilitate the integration of the VPU hardware decoding function into the Spice code, this application further organizes the VPU API and the VPU decoding process, classifies them as important functions, and compiles the VPU hardware decoding function into a dynamic link library libvpu.a, as shown in Table 1 specifically. Table 1 is the function composition of the decoding process.

[0079]

[0080]

[0081] Table 1

[0082] In Table 1, the functions after classification constitute the VPU hardware decoding function.

[0083] In Spice or other applications that call the libjpeg library, it is very convenient to use the above dynamic link library libvpu.a and the functions after classification to replace the Joint Photographic Experts Group (JPEG) software decoding part in the application. The JPEG software decoding part is the function composition in the open-source dynamic link library libvpu.a, thereby improving the performance of the application program.

[0084] Then, the host processor calls the dynamic link library to decode the image data of the current frame to be decoded, and obtains the decoded image of the current frame.

[0085] When using the dynamic link library libvpu.a, taking Spice as an example, first, the libvpu.a file needs to be placed in the corresponding lib folder during its installation. Then, modify the Makefile file to add -lvpu to the called library. Next, place the header files containing all the functions to be used in the compilation directory. Finally, replace the libjpeg functions with the functions after classification (VPU hardware decoding functions) in the Spice code.

[0086] Optionally, before the host processor sends the image data of the current frame to be decoded to the video processing unit VPU, so that the VPU decodes the image data of the current frame to be decoded through a preset decoding optimization method to obtain the decoded image of the current frame, the host processor sends the image processing requirement to the VPU through the Host Interface Reg register and receives the reply information corresponding to the image processing requirement forwarded by the VPU through the Host Interface Reg register.

[0087] Among them, the Host Interface Reg register is used to pass the commands of the host processor to the VPU and feedback the replies of the VPU to the host processor.

[0088] S302: When it is necessary to process the image data of the next frame to be decoded or an interrupt message sent by the VPU is received, the host processor obtains the decoded image of the current frame through a preset acquisition method and outputs it; the interrupt message is used to prompt that the image decoding operation of the image data of the current frame to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor itself.

[0089] In S302, the preset acquisition method is the shared method. The shared method is implemented based on the SDRAM Shared dynamic shared buffer.

[0090] SDRAM Shared is a dynamic buffer shared by the VPU and the host application, which is used to output the video frame data decoded by the VPU to the host application.

[0091] When it is necessary to process the image data of the next frame to be decoded or an interrupt message sent by the VPU is received, the decoded image of the current frame is obtained from the dynamic shared buffer and output; the decoded image of the current frame is sent by the VPU to the dynamic shared buffer.

[0092] Among them, the image data of the next frame to be decoded is the image data of the frame to be decoded at the next moment.

[0093] Obtaining the decoded image of the current frame from SDRAM Shared (in a shared manner) is the key to ensuring the minimum load of the host processor during decoding.

[0094] After the host processor sends the image data of the current frame to be decoded to the VPU and enables the VPU to compile the preset function into a dynamic link library, the host processor performs other data processing operations other than image data processing; the image data of the current frame to be decoded is sent during the execution of image data processing.

[0095] When processing the image data of the current frame to be decoded, there is no need for communication between the host processor and the VPU. The host processor only needs to send the image processing requirements to the VPU, and then the host processor can process other data processing operations until the host processor is ready to process the next frame of image or receives the interrupt message indicating that the data decoding by the VPU is completed, and then the decoded image is obtained from the VPU through SDRAM Shared. This is the key to ensuring the minimum load of the host processor during decoding.

[0096] The decoded images can form a video, and the video is mapped on the PKUnity86 platform for display.

[0097] Based on the PKUnity86 platform, after using the basic function of VPU hardware decoding, the present invention compares the total time consumption of using VPU decoding and using libjpeg decoding for images of different resolutions, and the comparison results are as

[0098] shown in Table 2.

[0099]

[0100] Table 2

[0101] In Table 2, the VPU decoding time is the decoding time of this solution, and the libjpeg decoding time is the software decoding time of the prior art.

[0102] As can be seen from Table 2, at the same image resolution, the decoding time of the VPU is less than that of the existing technology's libjpeg decoding time. Therefore, the decoding efficiency of the VPU is higher than that of the libjpeg software decoding efficiency.

[0103] The optimized VPU hardware decoding speed has an obvious advantage compared with the software decoding speed of libjpeg, solves the problem of video stuttering of Spice video mapping on the PKUnity86 platform, and improves the user experience.

[0104] Combined with the characteristics of the PKUnity86 platform, the multimedia mapping scheme of the remote display system Spice is optimized. By means of VPU hardware decoding and other means, the MJPEG decoding process is optimized, solves the stuttering problem existing in the current Spice video mapping, and improves the user experience.

[0105] In the embodiment of the present application, by adopting a preset decoding optimization method, that is, a hardware decoding method to decode the image data to be decoded, the decoded image data is obtained and output. Decoding by means of hardware decoding has a lower requirement for the computing power of the VPU, thereby improving the decoding efficiency of the image data to be decoded, and at the same time reducing the load of the host processor and avoiding the phenomenon of video stuttering.

[0106] The embodiment of the present application also provides a storage medium. The storage medium includes stored instructions. Among them, when the instructions run, the device where the storage medium is located is controlled to execute the above image data optimization method.

[0107] The embodiment of the present application also provides an electronic device. The schematic structural diagram is as Figure 4 shown, specifically including a memory 401 and one or more instructions 402. Among them, one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to execute the above image data optimization method.

[0108] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be adopted in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0109] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the corresponding description in the method embodiments.

[0110] The steps in the methods of the embodiments of the present application can be adjusted, combined, and deleted according to actual needs.

[0111] Finally, it should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0112] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0113] The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An image data optimization device, characterized in that, Applied to the PKUnity86 platform, the device includes: A host processor for sending the image data of the current frame to be decoded to the Video Processing Unit (VPU). The VPU is used to decode the image data of the current frame to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined by the VPU hardware decoding function. The host processor is further used to obtain and output the decoded image of the current frame by a preset acquisition method when it is necessary to process the image data of the next frame to be decoded or receive the interrupt message sent by the VPU; the interrupt message is used to indicate that the image decoding operation of the image data of the current frame to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor. The host processor is specifically further used to perform other data processing operations other than image data processing; the image data of the current frame to be decoded is sent when the host processor performs the image data processing. Wherein, the VPU is specifically used for: Compiling the VPU hardware decoding function into a dynamic link library; calling the dynamic link library to decode the image data of the current frame to be decoded to obtain the decoded image of the current frame. When using the dynamic link library libvpu.a, place the libvpu.a file in the corresponding lib folder during its installation, modify the Makefile file to add -lvpu to the called library, place the header files containing all the functions to be used in the compilation directory, and replace the libjpeg function with the VPU hardware decoding function in the Spice code.

2. The device according to claim 1, characterized in that, The host processor is specifically used for: When it is necessary to process the image data of the next frame to be decoded or receive the interrupt message sent by the VPU, obtain the decoded image of the current frame from the dynamic shared cache; the decoded image of the current frame is sent by the VPU to the dynamic shared cache.

3. The device according to claim 1, characterized in that, Specifically, the host processor is further used for: Sending image processing requirements to the VPU through the Host Interface Reg register and receiving the reply information corresponding to the image processing requirements sent by the VPU forwarded by the Host Interface Reg register.

4. An image data optimization method, characterized in that, Applied to the PKUnity86 platform, the method includes: Sending the image data of the current frame to be decoded to the Video Processing Unit (VPU) to enable the VPU to decode the image data of the current frame to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame; the preset decoding optimization method is determined by the VPU hardware decoding function. When it is necessary to process the image data of the next frame to be decoded or receive the interrupt message sent by the VPU, obtaining and outputting the decoded image of the current frame by a preset acquisition method; the interrupt message is used to indicate that the image decoding operation of the image data of the current frame to be decoded has been completed; the preset acquisition method is used to reduce the load of the host processor itself. Performing other data processing operations other than image data processing; the image data of the current frame to be decoded is sent during the execution of the image data processing. Among them, sending the current frame of image data to be decoded to the VPU, so that the VPU decodes the current frame of image data to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame, includes: Sending the current frame of image data to be decoded to the VPU, so that the VPU compiles the VPU hardware decoding function into a dynamic link library; Invoking the dynamic link library to decode the current frame of image data to be decoded to obtain the decoded image of the current frame. Among them, when using the dynamic link library libvpu.a, put the libvpu.a file into the corresponding lib folder during its installation, modify the Makefile file to add -lvpu to the called library, put the header files containing all functions to be used into the compilation directory, and replace the libjpeg function with the VPU hardware decoding function in the Spice code.

5. The method according to claim 4, characterized in that, When it is necessary to process the next frame of image data to be decoded or an interrupt message sent by the VPU is received, obtaining and outputting the decoded image of the current frame through a preset obtaining method, includes: When it is necessary to process the next frame of image data to be decoded or an interrupt message sent by the VPU is received, obtaining and outputting the decoded image of the current frame from the dynamic shared cache; the decoded image of the current frame is sent by the VPU to the dynamic shared cache.

6. The method according to claim 4, characterized in that, Before sending the current frame of image data to be decoded to the video processing unit VPU, so that the VPU decodes the current frame of image data to be decoded by a preset decoding optimization method to obtain the decoded image of the current frame, it further includes: Sending an image processing requirement to the VPU through the Host Interface Reg register, and receiving the reply information corresponding to the image processing requirement sent by the VPU forwarded by the Host Interface Reg register.

Images