Image processing method, cloud server, cloud desktop terminal, chip system and medium

Abstract

The invention discloses an image processing method, a cloud server, a cloud desktop terminal, a chip system and a medium, and the method comprises the steps: obtaining a to-be-synchronized grid associated with a change region of a current rendered image, and coding the image data of the to-be-synchronized grid according to preset coding information to obtain compressed image data; according to the grid image information of each to-be-synchronized grid, the compressed image data and the position in the current rendered image, respectively constructing each sub-image load; combining the sub-image loads and adding the coding information as image header information to obtain to-be-synchronized strip image information; and enabling the cloud desktop terminal to update a display image according to the to-be-synchronized strip image information, according to the method, only the to-be-synchronized grids needing to be synchronized are transmitted, the mode of transmitting a complete picture is replaced, and the data transmission amount is further reduced in combination with the mode of single image header information, so that the rendering speed is increased, and the image synchronization efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, a cloud server, a cloud desktop terminal, a chip system and a medium. Background Art

[0002] The cloud computing model is popular in modern enterprise IT (Internet Technology) architecture, that is, uploading all business and IT resources of the enterprise to the cloud. Cloud computing allows on-demand allocation and adjustment of computing resources to meet different workload requirements. Users can quickly expand or reduce computing power, storage capacity and network bandwidth as needed based on the services provided by cloud computing service providers without investing in and maintaining expensive physical infrastructure. Cloud computing enables users to access applications and data from anywhere and on any device. This provides flexibility for remote work, mobile office and collaboration, allowing users to work anytime and anywhere.

[0003] Now the C / S (server / client) architecture is used between the cloud server (virtual computer) and the cloud desktop terminal. The cloud server (virtual computer) adopts a composite encoding strategy to encode the computer screen of the virtual computer. Then it is transmitted to the cloud desktop terminal (client) via the Internet. The cloud desktop terminal decodes the compressed computer screen and displays it. The video layer of the cloud server (virtual computer) adopts video mode encoding (H264 / H265), and the other layers are synthesized and encoded in image mode (JPG).

[0004] There are also several problems with the current cloud server (virtual computer) and cloud desktop terminal architecture: (1) The communication between the cloud server and the cloud desktop terminal relies on an Internet connection and may be affected by network delays and bandwidth limitations. This may result in longer response times and may affect applications with high real-time requirements. (2) The cloud server (virtual computer) uses block-based JPG software encoding, and the cloud desktop terminal architecture uses block-based JPG software decoding. The cloud desktop terminal is under great pressure to decode a large number of JPG blocks. (3) In the image mode of the cloud server (virtual computer), the image content will change frequently, and the pressure on the cloud desktop terminal to decode JPG will increase significantly. In addition, network problems will be superimposed, and rendering freezes will become a prominent problem. Summary of the invention

[0005] The present invention provides an image processing method, a cloud server, a cloud desktop terminal, a chip system and a medium, which can improve the efficiency of image data synchronization.

[0006] In a first aspect of the present invention, an image processing method is provided. The method is applied to a cloud server and includes: obtaining a grid to be synchronized associated with a changed area of ​​a current rendered image, and encoding the image data of the grid to be synchronized according to preset encoding information to obtain compressed image data; constructing each sub-image load according to the grid image information of each grid to be synchronized, the compressed image data, and the position in the current rendered image; combining each sub-image load and adding the encoding information as image header information to obtain strip image information to be synchronized; and sending the strip image information to be synchronized to a cloud desktop terminal, so that the cloud desktop terminal updates the display image according to the strip image information to be synchronized.

[0007] In a second aspect of the present invention, an image processing method is provided. The method is applied to a cloud desktop terminal and includes: receiving image strip information to be synchronized, the image strip information to be synchronized includes image header information and sub-image loads of the grid to be synchronized associated with the changed area of ​​the rendered image; acquiring encoding information from the image header information of the image strip information to be synchronized; respectively acquiring compressed image data of each grid to be synchronized from each sub-image load of the combination of the image strip information to be synchronized; and decoding each compressed image data according to the encoding information to obtain image data of each grid to be synchronized.

[0008] In a third aspect of the present invention, a cloud server is provided, comprising: a memory configured to store an executable program; and a processor configured to execute the program to perform the image processing method as described above.

[0009] In a fourth aspect of the present invention, a cloud desktop terminal is provided. The cloud desktop terminal comprises: a memory configured to store an executable program; and a processor configured to execute the program to perform the image processing method as described above.

[0010] In a fifth aspect of the present invention, a chip system is provided, which includes: a processor configured to call and run an executable program from a memory, so that an electronic device equipped with the chip system executes the above-mentioned image processing method.

[0011] In a sixth aspect of the present invention, a computer readable medium is provided, on which a computer program is stored, wherein the computer program is executed to implement the image processing method as described above.

[0012] According to the present invention, only the image data corresponding to the grid to be synchronized is encoded according to the preset encoding information to obtain the compressed image data, and after combining the image loads corresponding to all the grids to be synchronized, an image header information is constructed to obtain the image information of the strip to be synchronized. Then, only one image header information is needed for the image information of the strip to be synchronized composed of multiple grids to be synchronized, which greatly reduces the amount of data to be transmitted, and only the grids to be synchronized that need to be synchronized are transmitted, instead of transmitting the complete picture, which further reduces the amount of data transmission, thereby accelerating the rendering speed and improving the efficiency of image synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of an image processing method applied to a cloud server according to an embodiment of the present invention;

[0014] Figure 2 A schematic diagram of the structure of a grid to be synchronized according to an embodiment of the present invention;

[0015] Figure 3 Flow chart of an image processing method applied to a cloud desktop terminal according to an embodiment of the present invention

[0016] Figure 4 A flowchart of obtaining a change area according to an embodiment of the present invention;

[0017] Figure 5 A flowchart of obtaining a grid to be synchronized according to an embodiment of the present invention;

[0018] Figure 6 A flowchart of constructing image information of strips to be synchronized according to an embodiment of the present invention;

[0019] Figure 7 A schematic diagram of the composition structure of the image information of the strip to be synchronized according to an embodiment of the present invention;

[0020] Figure 8 A flowchart of updating a displayed image according to an embodiment of the present invention;

[0021] Fig. 9 is a block diagram of an image processing system according to an embodiment of the present invention;

[0022] Fig.10 A schematic diagram of the structure of a cloud server according to an embodiment of the present invention;

[0023] Fig.11 The figure is a schematic diagram of the structure of a cloud desktop terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0025] In the existing technology, the desktop image synchronization method of the cloud desktop is usually to encode and transmit the entire computer screen, which takes a long time to encode and decode and requires a large amount of data to be transmitted. If the computer screen is split and then transmitted, an image header information needs to be stored for each part based on the existing encoding and decoding method, which has a limited effect on reducing the amount of transmitted data. The large amount of data will prolong the data transmission and encoding / decoding time, thereby increasing the delay of image synchronization or even causing freezing.

[0026] In order to solve at least the above-mentioned technical problems, the present disclosure provides an image processing method. According to the present disclosure, only the grids to be synchronized that need to be synchronized are processed to obtain sub-image loads, and a common image header information is generated for the sub-image loads corresponding to all grids to be synchronized. In this way, according to the embodiment of the present disclosure, when performing image synchronization, only part of the grid data to be synchronized can be transmitted without transmitting the complete current rendering image, and by combining the sub-image loads into the strip information to be synchronized, different grid data to be synchronized share one image header information, without the need to repeatedly transmit the image header information, thereby reducing the amount of transmitted data and the amount of data to be encoded / decoded.

[0027] According to some embodiments of the present disclosure, a non-standard JPG strip image encoder and decoder scheme is proposed, including a method for determining a local update of an image, a block division method of a JPG strip image, a storage format of a non-standard JPG strip image, an encoding process of a JPG strip image, a hardware decoding process of a JPG strip image, etc. The grid image overlapping with the changed area is encoded as a JPG strip image, thereby normalizing the irregular image into a unified form for convenience of subsequent processing. The JPG strip image is to reduce the redundancy of the image header information, and several image blocks share the image header information to reduce the redundancy of the image header information. Such processing can reduce the network transmission load to reduce the transmission delay, and reduce the JPG decoding load of the cloud desktop terminal to reduce the decoding delay. In addition, only the image of the changed area of ​​the rendered image is encoded as a JPG strip image, and the JPG strip image is transmitted to the terminal device, thereby saving bandwidth and providing transmission efficiency.

[0028] Hereinafter, the technical solution according to the present disclosure will be described with reference to specific embodiments and in conjunction with the accompanying drawings.

[0029] First, let’s explain the nouns:

[0030] JPG (JPEG) is a common image compression format, which contains two important components: DHT (Define Huffman Table) and Quantization Table.

[0031] DHT (Define Huffman Table): DHT is a table used for Huffman encoding and decoding in JPEG image compression. Huffman coding is a variable-length coding method used to map symbols (usually pixel values) in an image to a bit stream. The DHT table defines the codewords used in Huffman coding and the mapping relationship between the symbol values ​​represented by each codeword. The DHT table usually contains multiple Huffman coding tables, each table for a different component (such as brightness and chrominance) and different AC (alternating current) and DC (direct current) coefficients. By using the DHT table, JPEG compression can achieve efficient encoding, in which symbols that appear more frequently use shorter codewords, while symbols that appear less frequently use longer codewords.

[0032] Quantization table: The quantization table is a table used in JPEG compression to reduce image details. It divides the frequency domain data of the image (such as DCT coefficients) by the corresponding quantization factor and rounds the result to an integer. The quantization process suppresses high-frequency details, thereby reducing the complexity of the data and reducing the number of bits required for storage and transmission. The quantization table is usually an 8x8 matrix, in which each element represents the quantization factor of the DCT coefficient at the corresponding position. A higher quantization factor will result in a greater degree of detail loss, so using different quantization tables can control the quality and compression ratio of the image. In JPEG compression, the quantization table is customizable, allowing users to select the appropriate quantization factor as needed.

[0033] Figure 1 1 is a flowchart showing an image processing method 100 applied to a cloud server according to an embodiment of the present disclosure. Figure 1 The method 100 includes the following steps 102 to 108.

[0034] In step 102, a grid to be synchronized associated with a changed area of ​​a currently rendered image is acquired, and image data of the grid to be synchronized is encoded according to preset encoding information to obtain compressed image data.

[0035] In some embodiments, obtaining a grid to be synchronized associated with a changed region of a current rendered image includes: reading the current rendered image and a previous frame rendered image; calculating the changed region of the current rendered image compared to the previous frame rendered image; obtaining a grid image corresponding to the current rendered image; and setting a grid that overlaps with the changed region as a grid to be synchronized, the grid being the smallest unit of the grid image. In this way, only the grid where the changed region is located needs to be encoded and data transmitted, without the need to transmit the entire current rendered image; for the changed region, since the changed region is usually irregular, it is converted into a grid where the changed region exists, and finally the grid data where the changed region exists is encoded and transmitted, which is convenient for standardization.

[0036] In some embodiments, calculating the changed area of ​​the current rendered image compared to the previous rendered image includes: calculating the image residual value between the current rendered image and the previous rendered image; and marking the area in the current rendered image where the image residual value exceeds the residual threshold as the changed area. In this way, the definition of the changed area is quantified, and the image residual value between the current rendered image and the previous rendered image is calculated to identify the area where the residual value exceeds the residual threshold. The residual value can reflect whether there is a difference between the two frames of images, so that the changed area can be accurately divided, and the display effect can be guaranteed while reducing the amount of data transmission.

[0037] Specifically, Figure 2 is a schematic diagram showing the structure of a grid to be synchronized according to an embodiment of the present invention. Figure 2 In some embodiments, after calculating the change region of the current rendered image and the previous frame rendered image, the maximum bounding rectangle corresponding to the change region is obtained, and the length and width of the maximum bounding rectangle are parallel to the length and width of the grid in the grid image; the maximum bounding rectangle is used as the change region corresponding to the grid image. In this way, in the process of marking the grid that has an overlapping area with the change region as the grid to be synchronized, the change region can be marked by the coordinates of the vertices, and the grid in the corresponding grid image can be quickly obtained, which speeds up the efficiency of obtaining the grid to be synchronized.

[0038] In some embodiments, the encoding information includes defining a Huffman table and a quantization table, and encoding the image data of the to-be-synchronized grid according to the preset encoding information to obtain compressed image data includes: performing JPG encoding on the image data of the to-be-synchronized grid according to the defined Huffman table and the quantization table to obtain the compressed image data. In this way, the image is transmitted after being encoded by JPG, and JPG encoding has a wide range of applications and mature technology, and can achieve a good compression effect.

[0039] In step 104, each sub-image load is constructed according to the grid image information of each of the grids to be synchronized, the compressed image data, and the position in the current rendering image.

[0040] In some embodiments, the image length, image width and image height of each grid to be synchronized are obtained; and each sub-image load is constructed according to the image length, image width, image height, compressed image data and position in the current rendered image of each grid to be synchronized. In this way, in the process of transmitting multiple image loads together, the storage location of each image load can be obtained through the data length, and because the grid to be synchronized is not a complete current rendered image, storing its corresponding image width and height and position in the current rendered image can update the grid to be synchronized with sound changes in situ when performing image synchronization, and determine the position by the method of width and height + position point, without the need to store multiple coordinate point data, further reducing the data to be transmitted; at the same time, the compressed image data can be restored to the image data corresponding to the grid to be synchronized, and then the position where the image data should be rendered can be determined according to the stored image width and height and position.

[0041] In step 106, the sub-image payloads are combined and the encoding information is added as image header information to obtain the image strip information to be synchronized.

[0042] In some embodiments, the image header information is added to the first position of each of the combined sub-image payloads; and a strip image end mark is set at the last position of each of the combined sub-image payloads to obtain the strip image information to be synchronized. In this way, multiple sub-image payloads share one image header information, further reducing the amount of data transmission. In the process of decoding the strip image information to be synchronized, it can stop in time when the corresponding signal is obtained after the parsing is completed, further improving the decoding efficiency.

[0043] In step 108, the strip image information to be synchronized is sent to the cloud desktop terminal, so that the cloud desktop terminal updates the display image according to the strip image information to be synchronized.

[0044] In some embodiments, causing the cloud desktop terminal to update the displayed image according to the image information of the strip to be synchronized includes: causing the cloud desktop terminal to decode the compressed image data according to the encoding information in the image header information to obtain the image data of the grid to be synchronized; and causing the cloud desktop terminal to render the image data into the displayed image according to the image information and the position of the grid to be synchronized in the current rendered image. In this way, the decoding and rendering of the transmitted image information of the strip to be synchronized are realized, so that the displayed image can be updated normally.

[0045] Figure 3 2 is a flowchart showing an image processing method 200 applied to a cloud desktop terminal according to an embodiment of the present disclosure. Figure 3 The method 200 includes the following steps 202 to 208.

[0046] In step 202, the strip image information to be synchronized is received, wherein the strip image information to be synchronized includes image header information and sub-image loads of the grid to be synchronized associated with the changed area of ​​the rendered image.

[0047] In step 204, encoding information is obtained from the image header information of the strip image information to be synchronized. In some embodiments, the strip image information to be synchronized also includes a strip image end mark set at the end. In this way, the cloud desktop terminal can know when to stop parsing the strip image to be synchronized.

[0048] In step 206, compressed image data of each of the to-be-synchronized grids are respectively acquired from each of the sub-image loads of the combination of the to-be-synchronized strip image information.

[0049] In step 208, each of the compressed image data is decoded according to the coding information to obtain image data of each of the grids to be synchronized.

[0050] In some embodiments, the compressed image data are successively JPG decoded according to the coding information to obtain the image data. In this way, the compressed image data is encoded by the coding information in the image header information, so that the image data corresponding to each grid to be synchronized can be obtained, thereby realizing the restoration of the compressed image data.

[0051] In some embodiments, if the strip image end mark is obtained from the last bit of the strip image information to be synchronized after decoding each of the image data, the display image is updated according to each of the image data. In this way, after the strip image end mark is obtained, it means that the sub-image loads corresponding to each grid to be synchronized have been parsed. At this time, the display image is updated uniformly, which can avoid the cloud desktop terminal switching between the parsing and updating states, and reduce the hardware requirements for the cloud desktop terminal.

[0052] In some embodiments, updating the displayed image according to each of the image data includes: rendering each of the image data into the displayed image according to the grid image information of each of the grids to be synchronized and the position of each of the grids to be synchronized in the current rendered image. Specifically, rendering each of the image data into the displayed image according to the image length, image width and image height of each of the grids to be synchronized and the position of each of the grids to be synchronized in the current rendered image. In this way, the corresponding position in the displayed image can be rendered and updated according to the recorded position of the grid to be synchronized in the current rendered image, so that the complete current rendered image can be restored without transmitting the entire current rendered image, thereby reducing the amount of data transmission.

[0053] In some embodiments, decoding each compressed image data according to the coding information to obtain the image data of each grid to be synchronized includes: obtaining the sub-image loads in the strip image information to be synchronized one by one, decoding the compressed image data in the sub-image loads according to the coding information in the image header information to obtain the image data of the grid to be synchronized; and if the strip image end mark is obtained, performing the step of rendering the image data into the display image according to the image information and the position of the grid to be synchronized in the current rendering image. In this way, the sub-image loads in the strip information image to be synchronized are obtained one by one and decoded until the strip image end mark is obtained, avoiding missing the parsing of the image load, and after the image data in all the sub-image loads are obtained, that is, after the strip image end mark is parsed, the image is uniformly rendered, so that the rendering operation can be performed in a centralized manner without frequent switching.

[0054] In some embodiments, rendering the image data into the display image according to the image information and the position of the grid to be synchronized in the current rendering image includes: reading the image load corresponding to the grid to be synchronized according to the data length; rendering the image data into the display image according to the image width, image height and the position of the grid to be synchronized in the current rendering image. In this way, the decoding and rendering process of the image is realized, and the whole process of realizing image synchronization by transmitting the relevant information of the grid to be synchronized corresponding to the changed area is achieved.

[0055] Hereinafter, application scenarios of the image processing method according to the embodiment of the present invention will be described by way of examples.

[0056] Figure 4 is a flowchart showing an embodiment of the present invention to obtain a change area, referring to Figure 4 , this stage includes the following steps 302 to 308.

[0057] In step 302, the previous frame rendering image and the current rendering image are read.

[0058] In step 304, the image residual between the previous frame rendered image and the current rendered image is calculated. Specifically, the image residual between two adjacent frames of images is obtained by calculating the residual between the pixel values ​​of each pixel.

[0059] In step 306, when the image residual exceeds the residual threshold, the corresponding pixel point is marked as a change point.

[0060] In step 308, the finally obtained adjacent change points are combined into a change region, and the change region is divided into a number of rectangular image blocks, the length and width of the rectangular image blocks are parallel to the length and width of the grid of the grid image corresponding to the current rendering image.

[0061] Figure 5 FIG. 1 is a flowchart showing how to obtain a grid to be synchronized according to an embodiment of the present invention. Figure 5 , this stage includes the following steps 312 to 318.

[0062] In step 312, the current rendered image is divided into grid images.

[0063] In step 314, it is calculated whether the grid and the change region partially overlap, that is, whether there is an overlapping portion.

[0064] In step 316, a data container is used to store image data of grids to be synchronized that partially overlap.

[0065] In step 318, the stored image data of the grid to be synchronized is encoded according to the preset encoding information to obtain the strip image information to be synchronized. In some embodiments, the strip image information to be synchronized is JPG strip image information.

[0066] Figure 6 FIG. 4 is a flow chart showing the construction of image information of strips to be synchronized according to an embodiment of the present invention. Figure 7 Schematic diagram showing the composition structure of the image information of the to-be-synchronized strip according to an embodiment of the present invention. Figure 6 and Figure 7 , step 318 includes the following steps 321 to 326.

[0067] In step 321, the image header information in the to-be-synchronized strip image information is constructed. In some embodiments, the image header information includes a DHT (defined Huffman table) and a quantization table.

[0068] In step 322, the sub-image unit header information in the strip image information to be synchronized is constructed, each of the grids to be synchronized corresponds to a sub-image unit, and the sub-image unit header information includes data length, image width, image height, and position in the current rendered image. In some embodiments, the position of the sub-image unit in the current rendered image is represented by a coordinate point (X, Y), that is, the horizontal position / offset and vertical position / offset of the corresponding sub-image unit relative to the current rendered image, which can be defaulted to the position of a corner point of the grid to be synchronized corresponding to the sub-image unit.

[0069] In step 323, the corresponding encoder is called according to the encoding information to compress the sub-image data to obtain compressed image data, where the sub-image data is the image data corresponding to the grid to be synchronized.

[0070] In step 324, an image payload of the sub-image unit is constructed according to the sub-image unit header information and the compressed image data.

[0071] Steps 322 to 324 are repeatedly executed until all grids to be synchronized are converted.

[0072] In step 325, an end-of-image strip marker is written.

[0073] In step 326, the image information of the strip to be synchronized is transmitted to the end to be synchronized. In some embodiments, the end to be synchronized is a cloud desktop.

[0074] Assume that the current rendered image, that is, the image that the cloud server (virtual computer) needs to synchronize to the cloud desktop, is a screen with a resolution of 1920x1088, where the grid size is 32×32. If the entire current rendered image is transmitted, a total of 1920x1088) / (32x32)=2040 grids need to be encoded. Assume that the image header information size corresponding to a single grid JPG image is 600Bytes (bytes), and the compressed image data is 300Bytes; if the above method of constructing the strip image information to be synchronized is used, the maximum amount of image header information can be saved is 2040-1, which greatly improves the network transmission efficiency. That is, by setting only one image header information, the bandwidth corresponding to 2039 image header information is saved, thereby improving the transmission efficiency. Only the image of the changed area of ​​the rendered image is encoded as the strip image information to be synchronized, thereby saving bandwidth and providing transmission efficiency.

[0075] Figure 8 is a flowchart showing an embodiment of the present invention, referring to Figure 8 , this stage includes steps 331 to 336.

[0076] In step 331, the parser parses the image header information to obtain the DHT and quantization table.

[0077] At step 332, the decoder is configured according to the DHT and the quantization table.

[0078] At step 333, the decoder generates register parameters for the decoding process.

[0079] In step 334, the decoder decodes the compressed image data in the to-be-synchronized strip image information to obtain image data.

[0080] Steps 332-334 are repeatedly executed until the compressed image data in all sub-image units are decoded.

[0081] In step 335, the ISR interrupt is waited for when the image decoding is completed, that is, the strip image end mark is read, and a hardware interrupt is performed after the decoding is completed.

[0082] In step 336, the image rendering engine renders the image data to the corresponding position in the display image according to the image information and the position information, and completes the update of the display image. Specifically, the image rendering engine uses frame buffer off-screen rendering, and the decoded image data partially updates the off-screen buffer of the desktop image according to the image width, height and position information.

[0083] Fig. 9 4 is a block diagram showing an image processing system 400 according to an embodiment of the present invention. Fig. 9 , the image processing system 400 includes a cloud server and a cloud desktop terminal. The image encoding module 402 and the computing module 406 are arranged in the cloud server (virtual computer). The image decoding module 4042 and the rendering module 4044 are arranged in the cloud desktop terminal device. The cloud server and the cloud desktop terminal communicate with each other, for example, via the Internet or a data bus. In some embodiments, the rendering module is a strip image rendering engine, so as to realize desktop synchronization from the cloud server to the cloud desktop terminal device.

[0084] The cloud server includes an image encoding module 402 and a computing module 406. The image encoding module 402 is configured to obtain a grid to be synchronized that is associated with a changed area of ​​a currently rendered image, and to encode the image data of the grid to be synchronized according to preset encoding information to obtain compressed image data. The computing module 406 is configured to construct each sub-image load according to the grid image information of each grid to be synchronized, the compressed image data, and the position in the currently rendered image; combine each sub-image load and add the encoding information as image header information to obtain the strip image information to be synchronized; and send the strip image information to be synchronized to the cloud desktop terminal, so that the cloud desktop terminal updates the display image according to the strip image information to be synchronized.

[0085] It should be understood that the image encoding module 402 and the calculation module 406 can be further configured to execute the corresponding steps or actions in the image processing method as described above, which will not be described in detail here.

[0086] The cloud desktop terminal includes an image decoding module 4042 and a rendering module 4044. The image decoding module 4042 is configured to receive the strip image information to be synchronized, the strip image information to be synchronized includes image header information and sub-image loads of the grid to be synchronized associated with the changed area of ​​the rendered image; obtain encoding information from the image header information of the strip image information to be synchronized; respectively obtain compressed image data of each grid to be synchronized from each sub-image load of the combination of the strip image information to be synchronized; and decode each compressed image data according to the encoding information to obtain image data of each grid to be synchronized. The rendering module 4044 is configured to update the display image according to each image data.

[0087] It should be understood that the image decoding module 4042 and the rendering module 4044 can be further configured to execute the corresponding steps or actions in the image processing method as described above, which will not be described in detail here.

[0088] According to yet another aspect of the present invention, Fig.10 Schematic diagram of a cloud server 500 according to an embodiment of the present invention. Fig.10 The cloud server 500 includes a memory 502, a processor 504, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the image processing method implemented in the cloud server as described above is implemented.

[0089] According to yet another aspect of the present invention, Fig.11 is a schematic diagram showing a cloud desktop terminal 600 according to an embodiment of the present invention. Fig.11 The cloud desktop terminal 600 includes a memory 602, a processor 604, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the image processing method implemented in the cloud desktop terminal as described above is implemented.

[0090] According to another aspect of the present invention, a chip system is provided. The chip system includes a processor configured to call and run an executable program from a memory, so that an electronic device equipped with the chip system executes the image processing method implemented by a cloud server or an image processing method implemented by a cloud desktop terminal. In some embodiments, the chip system includes a system on chip SoC.

[0091] According to another aspect of the present invention, a computer-readable medium is provided, wherein a computer program is stored on the computer-readable medium, and the computer program is executed by a processor to implement the image processing method implemented in the cloud desktop terminal as described above.

[0092] In summary, the image processing method, cloud server, cloud desktop terminal, chip system and medium provided by the present invention determine the change area by calculating the residual value of the current rendered image and the previous frame rendered image, and obtain the grid image corresponding to the current rendered image, and only encode and transmit the grid image containing the change area, that is, the grid image overlapping with the change area, which reduces the amount of data in the transmission process, and also reduces the amount of data in the encoding and decoding process, thereby improving the efficiency of data synchronization refresh. In addition, in the process of encoding, for the image data corresponding to multiple grids to be synchronized, usually the encoding information is the same, so the same image header data is shared, further reducing the amount of data transmission. A strip image end mark is added to the strip image information to be synchronized, and the parsing can be automatically stopped after the parsing is completed, and the position and image width and height information are added to each image load in the strip image information to be synchronized, so that the position of the change area can be located, so that the change area is re-rendered on the displayed image, and the off-screen rendering method further improves the efficiency of image display.

[0093] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, applied to a cloud server, characterized in that: include: Acquire a to-be-synchronized grid associated with a changed region of a currently rendered image, and encode image data of the to-be-synchronized grid according to preset encoding information to obtain compressed image data; Constructing each sub-image load respectively according to the grid image information of each grid to be synchronized, the compressed image data and the position in the current rendering image; Combining the sub-image payloads and adding the encoding information as image header information to obtain the image strip information to be synchronized; as well as The strip image information to be synchronized is sent to the cloud desktop terminal, so that the cloud desktop terminal updates the display image according to the strip image information to be synchronized.

2. The image processing method according to claim 1, characterized in that: Obtaining the mesh to be synchronized associated with the changed area of ​​the current rendered image includes: Read the current rendered image and the previous frame rendered image; Calculating a changed area of ​​the current rendered image compared to the previous frame rendered image; Acquire a grid image corresponding to the current rendered image; and A grid having an overlapping area with the change area is set as a grid to be synchronized, and the grid is the smallest unit of the grid image.

3. The image processing method according to claim 2, characterized in that: Calculating the changed area of ​​the current rendered image compared to the previous frame rendered image includes: Calculating an image residual value between the current rendered image and the previous frame image; and Marking a region in the current rendered image where the image residual value exceeds a residual threshold as a changed region.

4. The image processing method according to claim 1, characterized in that: Constructing each sub-image load according to the grid image information of each grid to be synchronized, the compressed image data and the position in the current rendered image comprises: Obtaining the image length, image width and image height of each of the grids to be synchronized; and Each sub-image load is constructed respectively according to the image length, the image width, the image height, the compressed image data and the position in the current rendering image of each grid to be synchronized.

5. The image processing method according to claim 1, characterized in that: The step of causing the cloud desktop terminal to update the displayed image according to the strip image information to be synchronized comprises: The cloud desktop terminal decodes the compressed image data according to the coding information in the image header information to obtain image data of the grid to be synchronized; and The cloud desktop terminal renders the image data into a display image according to the image information and the position of the grid to be synchronized in the current rendering image.

6. The image processing method according to claim 1, characterized in that: Combining the sub-image payloads and adding the coding information as image header information to obtain the image information of the to-be-synchronized strip includes: Add the image header information to the first position of each of the combined sub-image payloads; and A strip image end mark is set at the end of each of the combined sub-image loads to obtain the strip image information to be synchronized.

7. The image processing method according to claim 1, characterized in that: The coding information includes defining a Huffman table and a quantization table, Encoding the image data of the grid to be synchronized according to preset encoding information to obtain compressed image data includes: performing JPG encoding on the image data of the grid to be synchronized according to the defined Huffman table and the quantization table to obtain the compressed image data.

8. An image processing method, applied to a cloud desktop terminal, characterized in that: include: receiving strip image information to be synchronized, wherein the strip image information to be synchronized includes image header information and a sub-image load of a grid to be synchronized associated with a changed area of ​​a rendered image; Acquire encoding information from the image header information of the to-be-synchronized strip image information; Respectively obtain the compressed image data of each of the to-be-synchronized grids from each of the sub-image loads of the combination of the to-be-synchronized strip image information; as well as The compressed image data are decoded according to the coding information to obtain the image data of each grid to be synchronized.

9. The image processing method according to claim 8, characterized in that: Decoding each of the compressed image data according to the coding information to obtain image data of each of the grids to be synchronized comprises: The compressed image data are successively JPG decoded according to the coding information to obtain the image data respectively.

10. The image processing method according to claim 9, characterized in that: The strip image information to be synchronized also includes a strip image end mark set at the last position, and the image processing method also includes: If the strip image end mark is obtained from the last bit of the strip image information to be synchronized after decoding and obtaining each of the image data, the display image is updated according to each of the image data.

11. The image processing method according to claim 10, characterized in that: Updating the displayed image according to each of the image data comprises: The image data are rendered into a display image according to the grid image information of each grid to be synchronized and the position of each grid to be synchronized in the current rendering image.

12. The image processing method according to claim 11, characterized in that: Rendering each of the image data into a display image according to the grid image information of each of the grids to be synchronized and the position of each of the grids to be synchronized in the current rendering image comprises: The image data are rendered into the display image according to the image length, image width and image height of each grid to be synchronized and the position of each grid to be synchronized in the current rendering image.

13. A cloud server, characterized in that: include: a memory configured to store an executable program; as well as A processor is configured to execute the program to perform the image processing method according to any one of claims 1 to 7.

14. A cloud desktop terminal, characterized in that: include: a memory configured to store an executable program; as well as A processor is configured to execute the program to perform the image processing method according to any one of claims 8 to 12.

15. A chip system, characterized in that: include: A processor is configured to call and run an executable program from a memory so that an electronic device equipped with the chip system executes the image processing method according to any one of claims 1 to 7 or the image processing method according to any one of claims 8 to 12.

16. A computer readable medium having a computer program stored thereon, characterized in that: The computer program is executed to implement the image processing method according to any one of claims 1 to 7 or the image processing method according to any one of claims 8 to 12.

Images