Image processing method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and system, and the method comprises the steps: obtaining an image processing instruction through a virtual equipment interface; acquiring color image data according to the image processing instruction, and encoding the color image data to obtain color image encoding data; and transmitting the color image coding data to a client, so that the client decodes and displays the color image coding data. According to the embodiment of the invention, complicated image acquisition and coding tasks are concentrated on the server side, so that the computing pressure of the client side is reduced, the overall performance of the cloud desktop system is optimized, and the use efficiency and the user experience of the cloud desktop system under limited networks and hardware resources are improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud desktop and coding and decoding technology, and in particular to an image processing method, an image processing system, an electronic device and a computer-readable storage medium. Background Art

[0002] With the continuous development of cloud computing and virtualization technology, cloud desktop technology has been widely used. The cloud desktop system concentrates the user's computing and storage tasks in the data center and transmits the desktop content to the client device in real time through remote transmission technology. In order to ensure the user's interactive experience, the cloud desktop system needs to achieve efficient image transmission, decoding and display under limited network bandwidth and terminal hardware resources.

[0003] In the cloud desktop system, the server needs to collect the desktop display content as image data, compress and encode the image data, and transmit it to the client through the network. The client decodes the encoded data after receiving it, and finally renders and displays the decoded image data.

[0004] However, due to the limitation of the hardware performance of the client, decoding and rendering the encoded data will occupy a large amount of central processing unit (CPU) resources, affecting the overall system performance of the client. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide an image processing method, an image processing system, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In order to solve the above problem, an embodiment of the present invention discloses an image processing method, which is applied to a server side. The method includes:

[0007] Get image processing instructions through the virtual device interface;

[0008] Collecting color image data according to the image processing instruction, and encoding the color image data to obtain color image encoding data;

[0009] The color image encoding data is transmitted to a client, so that the client decodes and displays the color image encoding data.

[0010] Optionally, the collecting color image data according to the image processing instruction and encoding the color image data to obtain color image encoded data includes:

[0011] Collect the color image data from the color channel according to the image processing instruction, the color image data including one of the following: RGB image data, YUV image data, CMYK image data;

[0012] The color image data is encoded to obtain the color image encoded data, where the color image encoded data includes one of the following: JPEG image encoded data, PNG image encoded data, and HEIC image encoded data.

[0013] Optionally, after acquiring the image processing instruction through the virtual device interface, the method further includes:

[0014] Disables collecting transparency image data from the transparency channel.

[0015] Optionally, transmitting the color image encoding data to a client so that the client decodes and displays the color image encoding data includes:

[0016] The color image encoding data is transmitted to the client, so that the client decodes the color image encoding data to obtain the color image data, creates a canvas object corresponding to the data format of the color image data, and displays the color image data on the canvas object.

[0017] The embodiment of the present invention further discloses an image processing method, which is applied to a client, and the method includes:

[0018] Acquire color image encoding data from the server, wherein the color image encoding data is obtained by encoding the collected color image data by the server according to the image processing instruction, and the image processing instruction is obtained by the server through the virtual device interface;

[0019] The color image coded data is decoded and displayed.

[0020] Optionally, decoding and displaying the color image encoded data comprises:

[0021] Decoding the color image coded data to obtain the color image data;

[0022] Creating a canvas object corresponding to the data format of the color image data;

[0023] The color image data is displayed on the canvas object.

[0024] The embodiment of the present invention further discloses an image processing system, which is applied to a server side. The system includes:

[0025] An image instruction acquisition module is used to acquire image processing instructions through a virtual device interface;

[0026] An image data encoding module, used for collecting color image data according to the image processing instruction, and encoding the color image data to obtain color image encoding data;

[0027] The coded data transmission module is used to transmit the color image coded data to the client, so that the client can decode and display the color image coded data.

[0028] Optionally, the image data encoding module includes:

[0029] A color image data acquisition module, used to acquire the color image data from the color channel according to the image processing instruction, the color image data including one of the following: RGB image data, YUV image data, CMYK image data;

[0030] The color image data encoding module is used to encode the color image data to obtain the color image encoding data, and the color image encoding data includes one of the following: JPEG image encoding data, PNG image encoding data, and HEIC image encoding data.

[0031] Optionally, the system further comprises:

[0032] The acquisition prohibition module is used to prohibit the acquisition of transparency image data from the transparency channel after the image instruction acquisition module acquires the image processing instruction through the virtual device interface.

[0033] Optionally, the encoded data transmission module is used to transmit the color image encoded data to the client, so that the client decodes the color image encoded data to obtain the color image data, creates a canvas object corresponding to the data format of the color image data, and displays the color image data on the canvas object.

[0034] The embodiment of the present invention further discloses an image processing system, which is applied to a client, and the system includes:

[0035] A coding data acquisition module, used for acquiring color image coding data from a server, wherein the color image coding data is obtained by encoding the collected color image data by the server according to an image processing instruction, and the image processing instruction is obtained by the server through a virtual device interface;

[0036] The coded data display module is used to decode and display the color image coded data.

[0037] Optionally, the coded data display module includes:

[0038] A color image coded data decoding module, used for decoding the color image coded data to obtain the color image data;

[0039] A canvas object creation module, used to create a canvas object corresponding to the data format of the color image data;

[0040] The color image data display module is used to display the color image data on the canvas object.

[0041] An embodiment of the present invention further discloses an electronic device, comprising: one or more processors; and one or more machine-readable media storing instructions thereon, which, when executed by the one or more processors, enable the electronic device to perform the image processing method as described above.

[0042] The embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the image processing method as described above.

[0043] The embodiments of the present invention include the following advantages:

[0044] The image processing solution provided by the embodiment of the present invention is applied to the server side. On the server side, an image processing instruction is obtained through a virtual device interface. Next, color image data is collected according to the image processing instruction, and the color image data is encoded to obtain color image encoded data. Then, the color image encoded data is transmitted to the client side, so that the client side decodes and displays the color image encoded data.

[0045] The embodiment of the present invention completes the acquisition and encoding of color image data on the server side, and directly transmits the color image encoding data to the client side, thereby avoiding the client side from performing complex image processing operations, reducing the computing burden of the client CPU, and being able to better adapt to terminal devices with limited hardware performance. The encoding processing of color image data is concentrated on the server side, and the client side only needs to decode the received color image encoding data, eliminating the original pixel-level splicing or complex data processing process, which can significantly improve the decoding efficiency of the client side and reduce the decoding delay. Moreover, by reducing the image processing tasks of the client side, the client side can allocate more computing resources to other key tasks, such as user interaction, data transmission management, etc., thereby improving the performance and user experience of the entire client side. Encoding color image data on the server side can more efficiently compress the amount of transmitted data, reduce the consumption of network bandwidth, and at the same time ensure the quality of transmitted data, thereby achieving a smoother image display effect under limited network conditions.

[0046] In general, the embodiments of the present invention reduce the computing pressure on the client side by concentrating complex image acquisition and encoding tasks on the server side, optimize the overall performance of the cloud desktop system, and improve the usage efficiency and user experience of the cloud desktop system under limited network and hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flowchart of a method for processing an image according to an embodiment of the present invention;

[0048] Figure 2 is a flowchart of another image processing method according to an embodiment of the present invention;

[0049] Figure 3 This is a working principle diagram of an image data processing solution based on a cloud desktop according to an embodiment of the present invention;

[0050] Figure 4 It is a flowchart of the steps of a cloud desktop-based image data processing solution according to an embodiment of the present invention;

[0051] Figure 5 is a structural block diagram of an image processing system according to an embodiment of the present invention;

[0052] Figure 6 It is a structural block diagram of another image processing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] The image processing scheme proposed in the embodiment of the present invention realizes efficient transmission and display of image data by centrally completing image acquisition and encoding processing on the server side. Specifically, the server side receives image processing instructions through the virtual device interface, acquires color image data according to the image processing instructions, and encodes it to generate color image encoding data. Subsequently, these color image encoding data are transmitted to the client, and the client only needs to decode the received color image encoding data and directly display it without the need for additional complex image processing operations. By transferring complex image processing tasks from the client to the server side, this solution significantly reduces the CPU burden of the client, improves decoding efficiency and cloud desktop system performance, and optimizes the utilization of network bandwidth, and is suitable for terminal devices with limited hardware resources.

[0055] Reference Figure 1 , shows a flowchart of the steps of an image processing method according to an embodiment of the present invention. The image processing method can be applied to a server. The image processing method can specifically include the following steps:

[0056] Step 101: Obtain image processing instructions through a virtual device interface.

[0057] In the embodiment of the present invention, the virtual device interface can be understood as an abstract layer between the server and the virtual machine of the cloud desktop system, which is used to transmit instructions or information related to image acquisition or processing. The image processing instruction may include specific processing tasks, such as the image area to be acquired, the acquisition time interval, the image format requirements, and whether compression processing is required.

[0058] Specifically, the virtual device interface can be implemented through the underlying virtualization technology of the virtual machine's operating system. Assume that in a cloud desktop scenario, a virtual machine instance is assigned to a user, and the user logs in to the instance through a remote client and operates the desktop. At this time, the virtual device interface captures the user's operation information and generates image processing instructions based on the operation. For example, when the user drags a window, the virtual device interface generates instructions to collect the image content of a specific area and update the corresponding frame buffer.

[0059] For example, suppose a user opens a picture on a virtual desktop and scales it. The virtual device interface can detect the change in window size, generate an image processing instruction, specify the image data of the zoomed display area, and pass the image processing instruction to the subsequent processing module.

[0060] Step 102: collect color image data according to the image processing instruction, and encode the color image data to obtain color image encoded data.

[0061] In an embodiment of the present invention, the server obtains the image processing instruction, determines the image area or frame buffer data to be collected, and parses it into color image data. The color image data is usually stored in a red-green-blue (RGB) format, including the color values ​​of all visible pixels on the screen.

[0062] After the acquisition is completed, the color image data needs to be encoded to generate compressed color image encoded data for transmission. The encoding method can use a coding format that supports a high compression ratio, such as the Joint Photographic Experts Group (JPEG), or a more efficient video encoding technology such as High Efficiency Video Coding (HEVC). Taking JPEG as an example, the encoding process includes block segmentation, color space conversion, discrete cosine transform, quantization, and entropy coding. Through encoding processing, the volume of color image data can be significantly reduced, saving bandwidth for subsequent transmission.

[0063] For example: Assume that the user's virtual desktop resolution is 1920×1080, and the volume of the collected original RGB image data is 1920×1080×3 bytes (about 5.93MB). After JPEG encoding, the RGB image data can be compressed to about 500KB, with a compression ratio of about 12:1. In this process, if the user drags the window and only affects part of the area, the server can collect only the color image data of the changed area according to the image processing instructions, and perform incremental encoding processing on it.

[0064] Step 103 : transmitting the color image encoding data to the client, so that the client decodes and displays the color image encoding data.

[0065] In the embodiment of the present invention, after the encoding process is completed, the server transmits the color image encoding data to the client through the network. The transmission method can adopt the Transmission Control Protocol / Internet Protocol (TCP / IP) to ensure the reliability and real-time performance of data transmission.

[0066] On the client side, the received color image encoding data will be decoded by the decoding module and restored to the original color image data. For example, if the transmitted color image encoding data is in JPEG format, the client will perform entropy decoding, inverse quantization, inverse transformation and color space restoration in sequence during decoding, and finally obtain RGB image data that can be directly rendered. The decoded RGB image data will be loaded into the graphics rendering module and displayed through a canvas object that supports the RGB format.

[0067] For example, if the server sends the collected 1920×1080 resolution color image data directly in RGB format, the client may not be able to display it smoothly due to the heavy decoding pressure. With the above processing method, the client only needs to decode the compressed JPEG data, for example, restore it from 500KB to RGB image data. In addition, the client can also combine the hardware decoding function to further improve the decoding speed and achieve low latency and high-quality image display effects.

[0068] The image processing solution provided by the embodiment of the present invention is applied to the server side. On the server side, an image processing instruction is obtained through a virtual device interface. Next, color image data is collected according to the image processing instruction, and the color image data is encoded to obtain color image encoded data. Then, the color image encoded data is transmitted to the client side, so that the client side decodes and displays the color image encoded data.

[0069] The embodiment of the present invention completes the acquisition and encoding of color image data on the server side, and directly transmits the color image encoding data to the client side, thereby avoiding the client side from performing complex image processing operations, reducing the computing burden of the client CPU, and being able to better adapt to terminal devices with limited hardware performance. The encoding processing of color image data is concentrated on the server side, and the client side only needs to decode the received color image encoding data, eliminating the original pixel-level splicing or complex data processing process, which can significantly improve the decoding efficiency of the client side and reduce the decoding delay. Moreover, by reducing the image processing tasks of the client side, the client side can allocate more computing resources to other key tasks, such as user interaction, data transmission management, etc., thereby improving the performance and user experience of the entire client side. Encoding color image data on the server side can more efficiently compress the amount of transmitted data, reduce the consumption of network bandwidth, and at the same time ensure the quality of transmitted data, thereby achieving a smoother image display effect under limited network conditions.

[0070] In general, the embodiments of the present invention reduce the computing pressure on the client side by concentrating complex image acquisition and encoding tasks on the server side, optimize the overall performance of the cloud desktop system, and improve the usage efficiency and user experience of the cloud desktop system under limited network and hardware resources.

[0071] In an exemplary embodiment of the present invention, color image data is collected according to an image processing instruction, and an implementation method of encoding the color image data to obtain color image encoded data is: color image data is collected from a color channel according to the image processing instruction. The color image data is encoded to obtain color image encoded data. The color image encoded data includes one of the following: JPEG image encoded data, PNG image encoded data, HEIC image encoded data, etc.

[0072] In practical applications, color image data refers to digital data that represents image color information, which usually includes the following types:

[0073] RGB image data: uses a weighted combination of the three primary colors of red, green, and blue to represent image color. It is the main color model for screen display devices.

[0074] Luminance-Chrominance (YUV) image data: The image color is represented by luminance and chrominance components. It is often used in video encoding because its component separation can reduce data redundancy.

[0075] Cyan-Magenta-Yellow-Black (CMYK) image data: a color model mainly used for printing, suitable for scenes that require precise color control.

[0076] During the acquisition process, the appropriate color model is selected according to the image processing instructions. For example, for cloud desktop environments, RGB image data is most commonly used because screen display devices are generally based on the RGB model, while for video stream processing, the YUV model is also widely used due to its compression friendliness.

[0077] For example, suppose a user opens a picture on a virtual desktop and rotates it. According to the image processing instruction, only the RGB image data of the rotated area is collected. For a video conference scenario, you may choose to collect YUV image data to reduce the amount of transmitted data while ensuring color accuracy.

[0078] The collected color image data is encoded to generate color image encoded data. The main purpose of encoding is to compress the image data volume and improve transmission and storage efficiency. Common encoding formats include:

[0079] JPEG image encoding data: uses a lossy compression algorithm, suitable for static images, with high compression efficiency, but may lose details.

[0080] Portable Network Graphics (PNG) image encoding data: supports lossless compression and transparent channels, suitable for high-quality image transmission.

[0081] High Efficiency Image Coding (HEIC) image coding data: Based on HEVC video coding technology, it supports high compression ratio and transparent channel, and is suitable for dynamic images or multi-frame image scenes.

[0082] The encoding process may include steps such as color space conversion, block processing, discrete cosine transform, quantization, and entropy coding. For example, for RGB data, it can be converted to YUV space first, and then the brightness and chrominance components are compressed separately to finally generate JPEG encoded data.

[0083] For example, if a user captures a static image with a resolution of 1920×1080, the original RGB data volume of the image is about 5.93MB. If you choose JPEG format encoding, the compressed data volume may be reduced to about 500KB, greatly reducing the cost of transmission and storage. For more efficient scenarios, you can choose HEIC encoding to further improve the compression ratio and obtain higher quality image performance.

[0084] The embodiment of the present invention supports multiple color models (such as RGB, YUV, CMYK) and encoding formats (such as JPEG, PNG, HEIC), making the cloud desktop highly flexible and able to select the optimal solution according to different application scenarios (such as cloud desktop, video conferencing, image processing, etc.). Through efficient encoding and processing of color image data, the data volume is significantly reduced, and smooth image transmission is achieved under conditions of limited network bandwidth, which is particularly suitable for remote display and real-time interaction scenarios.

[0085] In an exemplary embodiment of the present invention, after obtaining the image processing instruction through the virtual device interface, an implementation method is: prohibiting the collection of transparency image data from the transparency channel. This implementation optimizes the image processing and transmission process by prohibiting the collection of transparency image data from the transparency channel (such as the Alpha channel). Among them, the transparency channel is used to represent the transparency information of each pixel in the image, and its range is usually between 0 (completely transparent) and 255 (completely opaque), and is widely used in image synthesis, overlay effects and visual effects. However, in many application scenarios (such as cloud desktop display, video streaming, etc.), the data of the transparency channel may not be necessary. By prohibiting the collection of transparency image data from the transparency channel, the image processing process can be significantly simplified, the data volume and resource usage can be reduced, and it can adapt to scenarios with higher requirements for real-time and efficiency.

[0086] In actual application, after obtaining the image processing instruction through the virtual device interface, the image processing instruction is parsed to determine whether transparency information needs to be included. If it is clear that transparency image data is not needed, the transparency channel will be shielded and color image data containing only color information (such as RGB or YUV) will be collected. After shielding the transparency channel, only the color channel needs to be focused on. The structure of RGB image data is simpler, and the storage space occupied by each pixel is reduced by about 25%. This not only reduces the memory overhead of acquisition, but also reduces the computing resources required for subsequent encoding and transmission. Since the transparency image data is omitted, the transmitted image data is smaller in size, thereby reducing the consumption of network bandwidth.

[0087] For example: Assume that in a cloud desktop application scenario, the user operates the virtual machine desktop through a remote client. The user opens an application with a transparent window border (such as a semi-transparent console). In the traditional way, the image data of the transparency channel will also be collected and transmitted to the client. The client needs to decode and reconstruct the RGBA format image pixel by pixel (RGBA is a color space model consisting of RGB color space and Alpha channel), which not only increases the data transmission volume, but also occupies additional computing resources on the client.

[0088] In this embodiment, it is prohibited to collect transparency image data from the transparency channel, and only RGB image data is collected. At this time, the display effect of the transparent window in the desktop becomes completely opaque on the client. Although the transparency effect is lost, for ordinary user operation scenarios (such as document editing, web browsing), a completely opaque display is sufficient to meet functional requirements. And because RGB image data is smaller in volume than RGBA image data (reduced by about 25%), the transmission time is significantly shortened, and the delay felt by the user during operation is also greatly reduced. In addition, the client does not need to perform transparency synthesis operations, which further reduces the burden on the processor and improves fluency.

[0089] The embodiment of the present invention reduces the amount of data collected and encoded by about 25% by prohibiting the collection of transparency image data of the transparency channel, greatly reduces memory usage and network bandwidth consumption, improves data transmission efficiency, and is particularly suitable for scenarios with limited bandwidth. The client no longer needs to parse and synthesize transparency image data, the decoding and rendering processes are simplified, and the CPU and graphics processing unit (Graphics Processing Unit, GPU) resource consumption is significantly reduced. This is especially important for terminal devices with limited hardware performance (such as embedded systems or low-power processors). Due to the reduction in data volume and optimization of processing flow, the overall response speed of the cloud desktop is accelerated, the delay is reduced, and the user interaction experience is smoother. Although the transparency effect may be lost, in most non-visual special effects application scenarios, the opaque display is sufficient to meet user needs.

[0090] In an exemplary embodiment of the present invention, an implementation method of transmitting color image encoded data to a client so that the client decodes and displays the color image encoded data is: transmitting the color image encoded data to the client so that the client decodes the color image encoded data to obtain color image data, creating a canvas object corresponding to the data format of the color image data, and displaying the color image data on the canvas object.

[0091] In this implementation, after the server completes the encoding of the color image data, it transmits the encoded color image data to the client. After the client receives the color image data, it decodes the color image data to restore the original color image data. At the same time, the client creates a corresponding canvas object according to the data format of the color image data, and renders the color image data onto the canvas object for display.

[0092] In actual application, after receiving the color image encoding data, the client decodes the color image encoding data into color image data through the built-in or third-party decoding module. For example, JPEG data is decoded into a pixel value matrix in RGB format. After decoding, the client creates a matching canvas object (such as a texture buffer) according to the format and resolution of the color image data. The role of the canvas object is to provide a structured space for storage and rendering of color image data, supporting efficient graphics drawing operations.

[0093] After the canvas object is created, the client draws the decoded color image data pixel by pixel into the canvas object. The drawing process may involve color space conversion (such as from YUV to RGB) or resolution adjustment to ensure that the final display effect is consistent with the original image. By using hardware acceleration (such as GPU rendering), the client can achieve high-quality image display with lower resource overhead.

[0094] For example: suppose in a telemedicine system, doctors need to view high-definition medical images of patients in real time. The server collects medical image data through a high-resolution scanning device and performs JPEG encoding on it to reduce the data volume (such as compressing from 50MB to 1MB). The compressed JPEG image data is transmitted to the client. After receiving the JPEG image data, the client uses the built-in decoder to decode it and restore it to RGB image data with a resolution of 2048×2048. Subsequently, the client creates a canvas object of the same size as the image and fills the RGB image data into the canvas object for display. Doctors can view the details of medical images in real time through the client, and due to the use of compressed transmission and efficient decoding, it can maintain smoothness and high image quality under low bandwidth conditions.

[0095] The embodiment of the present invention significantly reduces the use of network bandwidth and improves data transmission efficiency by transmitting coded data instead of original image data. The client directly displays the coded data after decoding, which reduces the intermediate links of data processing and improves the real-time performance of the display. The client can dynamically create canvas objects according to different color data types RGB, which has strong versatility and adaptability.

[0096] Reference Figure 2 , shows a flowchart of another image processing method according to an embodiment of the present invention. The image processing method can be applied to a client. The image processing method can specifically include the following steps:

[0097] Step 201, obtaining color image encoding data from a server.

[0098] In an embodiment of the present invention, the color image encoding data is obtained by encoding the collected color image data according to the image processing instruction. The image processing instruction is obtained by the server through the virtual device interface. The specific content may include the resolution, acquisition area, encoding format (such as JPEG, PNG, HEIC, etc.) of the required image and related optimization parameters. After receiving the image processing instruction, the server will process and generate the corresponding color image encoding data as required, and the color image encoding data will be transmitted to the client over the network.

[0099] For example: Suppose a user accesses a virtual machine through a cloud desktop client and opens a high-resolution image in the virtual machine. The virtual machine sends an image processing instruction to the server through the virtual device interface, specifying the required image area and resolution (such as 1024×768). The server collects color image data according to the image processing instruction and compresses it into JPEG encoded data, and transmits it to the client in real time. After receiving it, the client enters the decoding and display stage.

[0100] Step 202: Decode and display the color image coded data.

[0101] In an embodiment of the present invention, after the client receives the color image encoding data, the color image encoding data is decoded and displayed. The main task of decoding is to restore the compressed color image encoding data to the original color image data. Subsequently, the client creates a canvas object corresponding to the color image data. The decoded color image data is filled into the canvas object and displayed on the client screen.

[0102] For example: In a cloud desktop scenario, the client decodes the received JPEG data and generates RGB image data. Subsequently, the client creates a 1024×768 pixel canvas object, fills the RGB image data into the canvas object pixel by pixel, and renders it through the browser. For video conferencing applications, the client receives the video stream (such as H.264 encoding), decodes the data into YUV format through the hardware decoder, and uses the GPU to render dynamic images on the screen in real time. Users can watch high-quality video conferencing content smoothly, while client resources are low.

[0103] In an exemplary embodiment of the present invention, an implementation method for decoding and displaying color image encoding data is: decoding the color image encoding data to obtain color image data; creating a canvas object corresponding to the data format of the color image data; and displaying the color image data on the canvas object.

[0104] Based on the above description of an embodiment of an image processing method, a cloud desktop-based image data processing solution is introduced below. Figure 3, showing a working principle diagram of an image data processing solution based on a cloud desktop according to an embodiment of the present invention.

[0105] The embodiment of the present invention starts from the server side and optimizes the original image encoding. Specifically, the data acquisition and encoding of the Alpha channel can be turned off, and only the JPEG image encoding data in RGB format can be transmitted. In this way, the server side reduces the processing and transmission of transparency information when performing image acquisition and encoding, thereby effectively reducing the volume of image data. On the client side of the cloud desktop, an RGB format canvas is used instead of the traditional RGBA format canvas. After receiving the RGB encoded image data, the client directly copies the decoded RGB image data to the RGB format canvas for display, avoiding the format conversion operation that was originally required. Through this optimization method, the client can directly render the RGB image data, reducing unnecessary calculations and memory usage, and greatly reducing the performance overhead caused by image format conversion. On the whole, this method makes the image transmission and display process more efficient, improves the image rendering performance of the client, especially on terminal devices with limited hardware resources, and can provide a smoother user experience.

[0106] Reference Figure 4 , showing a step flow chart of an image data processing solution based on a cloud desktop according to an embodiment of the present invention.

[0107] Step 401: The server obtains an image processing instruction through a virtual device interface.

[0108] The server receives specific image processing instructions through the virtual device interface. These image processing instructions are usually issued by the virtual machine of the cloud desktop and contain specific requirements for image acquisition and processing. For example, the image processing instructions may include detailed information such as the screen area to be acquired, image resolution, color format, encoding type, etc. The virtual device interface is a commonly used communication bridge in a virtualized environment, which can accurately transmit user requirements to the image processing module on the server side, thereby providing guidance for subsequent image acquisition and processing.

[0109] Step 402: The server collects RGB image data according to the image processing instruction.

[0110] After obtaining the image processing instruction, the server starts to collect the required RGB image data according to the content in the image processing instruction. RGB image data is an image representation composed of three color channels of red, green, and blue. It is a widely used standard image format that can clearly present the image content. The key to this step is to accurately collect the RGB image data of the specified area while ensuring the integrity and quality of the data, providing high-quality source data for subsequent encoding.

[0111] Step 403: The server encodes the RGB image data into JPEG image encoding data.

[0112] After collecting the RGB image data, the server will perform JPEG encoding on the RGB image data. JPEG is a widely used image compression encoding format that can significantly reduce the data volume through lossy compression while maintaining high visual quality. In this process, the server will select appropriate compression parameters according to specific needs, such as compression ratio, image quality level, etc., to generate JPEG image encoding data with moderate size and transmission-friendly. The main goal of this encoding step is to optimize image transmission efficiency, reduce network bandwidth usage, and ensure that the quality of the decoded image meets the display requirements.

[0113] Step 404: The server transmits the JPEG image encoding data to the client.

[0114] After JPEG encoding is completed, the server transmits the encoded JPEG image data to the client through the network. The transmission is usually based on a stable and efficient network protocol to ensure the integrity and accuracy of the data during the transmission process. In practical applications, in order to improve the transmission efficiency, the JPEG image encoding data may be further optimized, such as fragmented transmission, streaming transmission, etc., especially when the network conditions are complex or the transmission delay is large. This step is the key link in connecting the server and the client in the image processing solution, which directly affects the real-time performance and response speed of the overall system.

[0115] Step 405: The client decodes the JPEG image encoding data into RGB image data.

[0116] After receiving the JPEG image encoding data transmitted by the server, the client first decodes the JPEG image encoding data through the decoding module and restores it to the original RGB image data. The decoding process will restore the compressed image encoding format to pixel-level data representation, thus providing a basis for image rendering and display. Decoding is usually completed by a software decoder or a hardware decoding module (such as GPU acceleration), and the specific choice depends on the client's hardware capabilities and performance requirements. After decoding, the RGB image data is stored in a structured manner, which is convenient for subsequent canvas operations and displays.

[0117] Step 406: The client creates a canvas in RGB format and displays the RGB image data on the canvas.

[0118] After decoding the RGB image data, the client creates a corresponding RGB canvas according to the format and resolution of the image data. The canvas is the carrier for displaying image data, which is usually implemented based on the graphics rendering interface of the client device. After the canvas is created, the client directly fills the RGB image data into the canvas for display. No additional format conversion operations are required in this process, which maximizes the display efficiency and ensures that the image content can be presented to the user quickly and accurately.

[0119] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0120] Reference Figure 5 , shows a structural block diagram of an image processing system according to an embodiment of the present invention. The image processing system is applied to a server side, and the image processing system may specifically include the following modules.

[0121] An image instruction acquisition module 51 is used to acquire image processing instructions through a virtual device interface;

[0122] An image data encoding module 52, configured to collect color image data according to the image processing instruction, and encode the color image data to obtain color image encoded data;

[0123] The coded data transmission module 53 is used to transmit the color image coded data to the client, so that the client can decode and display the color image coded data.

[0124] In an exemplary embodiment of the present invention, the image data encoding module 52 includes:

[0125] A color image data acquisition module, used to acquire the color image data from the color channel according to the image processing instruction, the color image data including one of the following: RGB image data, YUV image data, CMYK image data;

[0126] The color image data encoding module is used to encode the color image data to obtain the color image encoding data, and the color image encoding data includes one of the following: JPEG image encoding data, PNG image encoding data, and HEIC image encoding data.

[0127] In an exemplary embodiment of the present invention, the system further comprises:

[0128] The acquisition prohibition module is used to prohibit the acquisition of transparency image data from the transparency channel after the image instruction acquisition module 51 acquires the image processing instruction through the virtual device interface.

[0129] In an exemplary embodiment of the present invention, the encoded data transmission module 53 is used to transmit the color image encoded data to the client, so that the client decodes the color image encoded data to obtain the color image data, creates a canvas object corresponding to the data format of the color image data, and displays the color image data on the canvas object.

[0130] Reference Figure 6 , shows a structural block diagram of another image processing system according to an embodiment of the present invention. The image processing system is applied to a client, and the image processing system may specifically include the following modules.

[0131] The coded data acquisition module 61 is used to acquire color image coded data from the server, wherein the color image coded data is obtained by the server encoding the collected color image data according to the image processing instruction, and the image processing instruction is obtained by the server through the virtual device interface;

[0132] The coded data display module 62 is used to decode and display the color image coded data.

[0133] In an exemplary embodiment of the present invention, the coded data display module 62 includes:

[0134] A color image coded data decoding module, used for decoding the color image coded data to obtain the color image data;

[0135] A canvas object creation module, used to create a canvas object corresponding to the data format of the color image data;

[0136] The color image data display module is used to display the color image data on the canvas object.

[0137] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0138] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0139] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0140] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0143] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0144] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0145] The above is a detailed introduction to an image processing method and an image processing system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing an image, characterized in that: Applied to the server side, the method includes: Get image processing instructions through the virtual device interface; Collecting color image data according to the image processing instruction, and encoding the color image data to obtain color image encoding data; The color image encoding data is transmitted to a client, so that the client decodes and displays the color image encoding data.

2. The method according to claim 1, characterized in that The step of collecting color image data according to the image processing instruction and encoding the color image data to obtain color image encoded data includes: Collect the color image data from the color channel according to the image processing instruction, the color image data including one of the following: RGB image data, YUV image data, CMYK image data; The color image data is encoded to obtain the color image encoded data, where the color image encoded data includes one of the following: JPEG image encoded data, PNG image encoded data, and HEIC image encoded data.

3. The method according to claim 2, characterized in that After acquiring the image processing instruction through the virtual device interface, the method further includes: Disables collecting transparency image data from the transparency channel.

4. The method according to claim 1, characterized in that The transmitting the color image encoding data to the client so that the client decodes and displays the color image encoding data includes: The color image encoding data is transmitted to the client, so that the client decodes the color image encoding data to obtain the color image data, creates a canvas object corresponding to the data format of the color image data, and displays the color image data on the canvas object.

5. A method for processing an image, characterized in that: Applied to a client, the method comprises: Acquire color image encoding data from the server, wherein the color image encoding data is obtained by encoding the collected color image data by the server according to the image processing instruction, and the image processing instruction is obtained by the server through the virtual device interface; The color image coded data is decoded and displayed.

6. The method according to claim 5, characterized in that The decoding and displaying of the color image coded data comprises: Decoding the color image coded data to obtain the color image data; Creating a canvas object corresponding to the data format of the color image data; The color image data is displayed on the canvas object.

7. An image processing system, characterized in that: Applied to the server side, the system includes: An image instruction acquisition module is used to acquire image processing instructions through a virtual device interface; An image data encoding module, used for collecting color image data according to the image processing instruction, and encoding the color image data to obtain color image encoding data; The coded data transmission module is used to transmit the color image coded data to the client, so that the client can decode and display the color image coded data.

8. An image processing system, characterized in that: Applied to a client, the system comprises: A coding data acquisition module, used for acquiring color image coding data from a server, wherein the color image coding data is obtained by encoding the collected color image data by the server according to an image processing instruction, and the image processing instruction is obtained by the server through a virtual device interface; The coded data display module is used to decode and display the color image coded data.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the electronic device to perform the image processing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the image processing method as claimed in any one of claims 1 to 6.

Images