Data transmission method, data transmission system, electronic device, and storage medium

Abstract

The application discloses a data transmission method, a data transmission system, an electronic device and a storage medium. The method comprises the following steps: a remote desktop server sends a plurality of drawing instructions to a relay server, wherein the drawing instructions comprise bitmap instructions and non-bitmap instructions; in the process of drawing a first image according to the plurality of drawing instructions, the relay server triggers an encoding operation on a first drawn image to obtain encoding data according to a bitmap instruction when receiving one bitmap instruction, and does not perform an encoding operation on a second drawn image according to a non-bitmap instruction when receiving one non-bitmap instruction; the first drawn image is an image that has been drawn in a canvas after drawing according to the bitmap instruction; the second drawn image is an image that has been drawn in the canvas after drawing according to the non-bitmap instruction; the encoding data and the non-bitmap instruction are sent to a client; the client decodes the encoding data to obtain decoding data, and draws an image according to the decoding data and the non-bitmap instruction. The embodiment improves the picture definition.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a data transmission method, a data transmission system, an electronic device, and a storage medium. Background Technology

[0002] With the increasing popularity of remote work, remote desktop technology is being used more and more widely. Remote desktop technology can be understood as allowing users on the other end of a network to receive the desktop information displayed on a computer after enabling remote desktop connection, and to operate that computer. The implementation of remote desktop technology enables remote users to access other desktops or applications from client devices via a local area network (LAN) or wide area network (WAN) to a server.

[0003] However, in remote desktop applications, problems such as low remote desktop operation smoothness and unclear picture quality are common. There is an urgent need for solutions to improve the low remote desktop operation smoothness and picture quality. Summary of the Invention

[0004] This application provides a data transmission method, a data transmission system, an electronic device, and a storage medium to solve the problems of low remote desktop operation flow and unclear screen in remote desktop applications.

[0005] In a first aspect, embodiments of this application provide a data transmission method, including:

[0006] The remote desktop server splits the target image to be sent into multiple drawing instructions and sends the multiple drawing instructions to the relay server. The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images.

[0007] The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing process, when a bitmap instruction is received, the server triggers an encoding operation on the first drawn image to obtain encoded data. When a non-bitmap instruction is received, the server does not perform an encoding operation on the second drawn image. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction.

[0008] The relay server sends the encoded data and the non-bitmap instructions to the client;

[0009] The client decodes the encoded data to obtain decoded data, and draws an image based on the decoded data and the non-bitmap instructions.

[0010] Secondly, embodiments of this application provide a data transmission system, including a remote desktop server, a relay server, and a client; wherein,

[0011] The remote desktop server is used to split the target image to be sent into multiple drawing instructions, and send the multiple drawing instructions to the relay server, wherein the drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images;

[0012] The relay server is used to draw a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing of the first image, when a bitmap instruction is received, an encoding operation is triggered on the first drawn image according to the bitmap instruction to obtain encoded data. When a non-bitmap instruction is received, no encoding operation is performed on the second drawn image, and the encoded data and the non-bitmap instruction are sent to the client. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction.

[0013] The client is used to decode the encoded data to obtain decoded data, and to draw an image based on the decoded data and the non-bitmap instructions.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] The solution provided in this application embodiment involves a remote desktop server splitting a target image to be sent into multiple drawing instructions and sending these instructions to a relay server. The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images. The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing process, upon receiving a bitmap instruction, it triggers an encoding operation on the first drawn image to obtain encoded data. Upon receiving a non-bitmap instruction, it does not perform an encoding operation on a second drawn image. The first drawn image is the image completed on the canvas after drawing according to the bitmap instructions, and the second drawn image is the image completed on the canvas after drawing according to the non-bitmap instructions. The relay server sends the encoded data and the non-bitmap instructions to the client. The client decodes the encoded data to obtain... The system decodes data and draws images based on the decoded data and the non-bitmap instructions. This allows the relay server to only trigger encoding of the already drawn image on the canvas when it receives a bitmap instruction, and not to encode the already drawn image on the canvas when it receives a non-bitmap instruction. Compared to related technologies where encoding is required for every drawing instruction, this reduces the number of encoding operations, lowers the consumption of encoding resources and time, thereby improving the smoothness and clarity of the remote desktop operation. Furthermore, the client decodes the encoded data to obtain decoded data. Because the relay server reduces the number of encoding operations, the client also reduces the number of decoding operations, lowering the client's consumption of decoding resources and time consumption, further improving the smoothness and clarity of the remote desktop operation. Additionally, the client draws images based on the decoded data and non-bitmap instructions. Since the non-bitmap instructions are not lost during transmission, this further ensures image clarity. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the data transmission method in an embodiment of this application;

[0018] Figure 2 This is a flowchart illustrating yet another data transmission method in an embodiment of this application;

[0019] Figure 3 This is a flowchart illustrating the data transmission system in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] The data transmission method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0024] Figure 1 A schematic diagram of the steps of a data transmission method in one embodiment of the present invention is shown. The method includes the following steps:

[0025] Step 101: The remote desktop server splits the target image to be sent into multiple drawing instructions and sends the multiple drawing instructions to the relay server.

[0026] The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images.

[0027] Specifically, the remote desktop server can split the target image into bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images through the virtual graphics card driver and the virtual graphics card device.

[0028] Non-bitmap images include text or icons.

[0029] Bitmap images, also known as raster or pixel images, are composed of individual dots called pixels. Furthermore, icons can be miniature icons.

[0030] Since bitmap images require more transmission resources during transmission, while text and icons require less, this embodiment can distinguish multiple drawing instructions into bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images based on the amount of resources required for transmission, so as to provide data support for subsequent selection of encoding objects.

[0031] Specifically, the remote desktop server can send multiple drawing commands to the relay server in a preset linear order to ensure that it is easy to detect whether any drawing commands are lost during the sending process.

[0032] Step 102: The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. During the process of drawing the first image, when a bitmap instruction is received, the server triggers an encoding operation on the first drawn image to obtain encoded data according to the bitmap instruction. When a non-bitmap instruction is received, the server does not perform an encoding operation on the second drawn image.

[0033] Specifically, the relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. For example, assuming the drawing instructions include A, B, C, and D, the relay server draws the first image on the pre-created canvas according to the drawing instructions A, B, C, and D. That is, the first image is composed of the drawing instructions A, B, C, and D.

[0034] During the drawing of the first image, each time the relay server receives a bitmap instruction, it triggers an encoding operation on the first drawn image according to the bitmap instruction to obtain encoded data. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction. That is to say, each bitmap instruction will trigger an encoding operation once, and the encoding operation triggered by each bitmap instruction is for the image that has been drawn on the canvas after drawing using the bitmap instruction and the drawing instructions received before receiving the bitmap instruction. For example, continuing the previous example, assuming drawing instruction C is a bitmap instruction, upon receiving drawing instruction C, an encoding operation is triggered on the first drawn image to obtain encoded data. If the relay server has received drawing instructions A and B before receiving drawing instruction C, then the first drawn image is the image drawn using drawing instructions A, B, and C. As another example, assuming drawing instruction D is a bitmap instruction, upon receiving drawing instruction D, an encoding operation is triggered on the first drawn image to obtain encoded data. If the relay server has received drawing instructions A, B, and C before receiving drawing instruction D, then the first drawn image is the image drawn using drawing instructions A, B, C, and D. It can be seen that since each received bitmap instruction triggers an encoding operation to obtain one piece of encoded data, the number of encoded data pieces ultimately obtained by the relay server is the same as the number of bitmap instructions.

[0035] Furthermore, during the process of drawing the first image, no encoding operation is performed on the second drawn image upon receiving each non-bitmap instruction. The second drawn image is the image already drawn on the canvas after drawing according to the non-bitmap instruction. In other words, each non-bitmap instruction does not trigger an encoding operation; that is, upon receiving each non-bitmap instruction, only drawing according to that instruction is required, without encoding the second drawn image. In this case, the second drawn image is the image jointly drawn using the non-bitmap instruction and the drawing instructions received before receiving the non-bitmap instruction. For example, continuing the above example, assuming drawing instruction B is a non-bitmap instruction, no encoding is performed on the second drawn image upon receiving drawing instruction B. If the relay server has received drawing instruction A before receiving drawing instruction B, then the second drawn image is the image drawn using drawing instructions A and B together.

[0036] In this way, the relay server only needs to trigger the encoding operation of the already drawn image on the canvas when it receives a bitmap command, and does not need to encode the already drawn image on the canvas when it receives a non-bitmap command. Compared with related technologies that require an encoding operation for every drawing command received, this reduces the number of encoding operations by not triggering encoding for every drawing command. Furthermore, the relay server only needs to encode the already drawn image and does not need to encode the blank areas on the canvas, which reduces the occupation of encoding resources and time consumption, thereby improving the smoothness of operation and the clarity of the screen on the remote desktop.

[0037] In addition, since bitmap instructions require a large amount of transmission resources and bandwidth during transmission, the relay server triggers encoding operations through bitmap instructions to obtain encoded data, thereby reducing the bandwidth required during data transmission.

[0038] Step 103: The relay server sends the encoded data and the non-bitmap instructions to the client.

[0039] Specifically, after the relay server obtains encoded data equal to the number of bitmap instructions, it can send the encoded data and non-bitmap instructions to the client.

[0040] Step 104: The client decodes the encoded data to obtain decoded data, and draws an image based on the decoded data and the non-bitmap instructions.

[0041] Specifically, the client only decodes the encoded data to obtain the decoded data, without needing to decode the non-bitmap instructions. Because the number of encoding operations on the relay server is reduced, the number of times the client decodes is reduced, thus reducing the client's consumption of decoding resources and time consumption, and improving the smoothness of operation and image clarity of the remote desktop. In addition, the client draws images based on the decoded data and non-bitmap instructions. Since the non-bitmap instructions are not lost during transmission, the image clarity of the image displayed by the client is further guaranteed.

[0042] In this embodiment, the relay server only needs to trigger the encoding operation of the already drawn image on the canvas when it receives a bitmap instruction, and does not need to encode the already drawn image on the canvas when it receives a non-bitmap instruction. Compared with related technologies where an encoding operation is required for each drawing instruction received, this reduces the number of encoding operations, the occupation of encoding resources, and the time consumption, thereby improving the smoothness of operation and the clarity of the screen on the remote desktop. In addition, the client decodes the encoded data to obtain decoded data. Since the number of encoding operations by the relay server is reduced, the number of decoding operations by the client is also reduced, which reduces the occupation of decoding resources and the time consumption of the client, thus improving the smoothness of operation and the clarity of the screen on the remote desktop. Furthermore, the client draws images based on the decoded data and non-bitmap instructions. Since the non-bitmap instructions are not lost during transmission, the clarity of the screen is further guaranteed.

[0043] In one embodiment, when the relay server draws a first image on a pre-created canvas according to the plurality of drawing instructions, the process includes the following steps:

[0044] The relay server draws the first image on the canvas in a linear order according to the received drawing instructions;

[0045] Upon receiving a bitmap instruction, an encoding operation is triggered on the first drawn image according to the bitmap instruction to obtain encoded data. Upon receiving a non-bitmap instruction, no encoding operation is performed on the second drawn image, including:

[0046] Detect whether the currently received drawing instruction is the bitmap instruction;

[0047] If the currently received drawing instruction is a bitmap instruction, then drawing is performed according to the bitmap instruction, and after drawing, an encoding operation is triggered on the first drawn image to encode the first drawn image and obtain encoded data.

[0048] If the currently received drawing instruction is a non-bitmap instruction, then drawing is performed according to the non-bitmap instruction, and no encoding operation on the second drawn image is triggered after drawing.

[0049] Specifically, the remote desktop server can send multiple drawing commands to the relay server in a preset linear order. The relay server can then draw the first image on the canvas according to the received drawing commands in that linear order. For example, continuing the previous example, assuming the receiving order of drawing commands A, B, C, and D is A, B, C, D, the relay server will draw on the canvas first according to drawing command A, then according to drawing command B, then according to drawing command C, and finally according to drawing command D.

[0050] In addition, specifically, each time the relay server receives a drawing instruction, it can detect whether the currently received drawing instruction is a bitmap instruction. If it is a bitmap instruction, it draws according to the bitmap instruction, and after drawing, it triggers the encoding operation of the first drawn image to encode the first drawn image to obtain encoded data, and can cache the encoded data in a preset sending queue. If it is a non-bitmap instruction, it draws according to the non-bitmap instruction, and after drawing, it does not trigger the encoding operation of the second drawn image, and can cache the non-bitmap instruction in a preset sending queue.

[0051] By determining whether each received drawing instruction is a bitmap instruction, it is possible to determine whether an encoding operation is triggered. Furthermore, the encoding operation triggered by the received bitmap instruction and the drawing operation based on the currently received drawing instruction can be performed in parallel, reducing the time loss of data transmission.

[0052] Furthermore, in one embodiment, the relay server sends the encoded data and the non-bitmap instructions to the client, including:

[0053] The relay server sends the encoded data and the non-bitmap instructions to the client in a linear order of receiving the multiple drawing instructions, wherein the position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

[0054] Specifically, the relay server can send encoded data and non-bitmap instructions to the client in a linear order of receiving multiple drawing instructions. Since each bitmap instruction triggers an encoding operation to obtain encoded data, the position of the encoded data in the linear order can be the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order. This ensures the consistency of the transmission order during data transmission, so that the relay server does not need to change the position of each data, which facilitates the relay server's data transmission.

[0055] In another embodiment, the non-bitmap instruction carries coordinate position information of the non-bitmap image in the client's canvas;

[0056] When the client draws an image based on the decoded data and the non-bitmap instruction, the client draws a second image on a pre-drawn canvas based on the decoded data; and according to the coordinate position information carried in the non-bitmap instruction, the non-bitmap image is superimposed on the area of ​​the second image corresponding to the coordinate position information.

[0057] Specifically, after receiving encoded data and non-bitmap instructions, the client can first select encoded data from the received data and decode it to obtain decoded data. Then, based on the decoded data, a second image is drawn on a pre-drawn canvas on the client. After drawing the second image, the non-bitmap image in the non-bitmap instruction is superimposed on the area corresponding to the coordinate position information in the second image according to the coordinate position information, thus obtaining the final image. This makes the non-bitmap image displayed by the client clearer and improves the image clarity.

[0058] In addition, it should be noted that after the client draws the second image on the canvas, it can read the image data and display it, thereby realizing the display of the target image.

[0059] The following is through Figure 2 The data transmission process between the relay server and the client is explained.

[0060] Specifically, the remote desktop server uses a virtual graphics card device to convert the target image to be transmitted into various drawing instructions, including bitmap instructions carrying bitmap images, non-bitmap instructions carrying text, and non-bitmap instructions carrying miniature icons.

[0061] The relay server creates a drawing canvas based on the display resolution. Then, the relay server draws the image on the canvas in a linear order according to the received drawing instructions. In other words, whether it is a non-bitmap instruction or a bitmap instruction, an image needs to be drawn, thereby ensuring that the resolution of the encoded bitmap is fixed, which in turn ensures the stability of subsequent encoding.

[0062] Then, during the image drawing process, the relay server checks whether the currently received drawing instruction is a bitmap instruction. If the currently received drawing instruction is a bitmap instruction, drawing is performed according to the bitmap instruction, and after drawing, an encoding operation is triggered on the first drawn image. The drawn image is read and sent to the encoding queue and then sent to the encoder. The encoder encodes the first drawn image to obtain encoded data, and the encoded data is cached in a preset sending queue. If the currently received drawing instruction is a non-bitmap instruction, drawing is performed according to the non-bitmap instruction, and after drawing, no encoding operation is triggered on the second drawn image. The non-bitmap instruction is cached in a preset sending queue. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction.

[0063] Then, the relay server sends the encoded data and non-bitmap instructions to the client in a linear order of receiving multiple drawing instructions. The position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

[0064] Next, the client receives the data sent by the relay server and determines whether the received data is encoded data. If so, the encoded data is decoded to obtain decoded data, and a second image is drawn on the canvas pre-drawn by the client based on the decoded data. If it is a non-bitmap instruction, the non-bitmap image is superimposed on the area corresponding to the coordinate position information of the non-bitmap image in the client's canvas according to the coordinate position information of the non-bitmap image carried in the non-bitmap instruction, to obtain the final drawn image.

[0065] It should be noted that this embodiment can also cache data before drawing. Specifically, if the data currently being traversed by the client is encoded data, the encoded data is decoded to obtain decoded data and then cached. If the data currently being traversed is a non-bitmap instruction, it is cached, and then the data is retrieved from the cache and the image is drawn.

[0066] Finally, the client reads the drawn image and displays it on the screen.

[0067] In this embodiment, the relay server only needs to trigger the encoding operation of the already drawn image on the canvas when it receives a bitmap instruction. It does not need to encode the already drawn image on the canvas when it receives a non-bitmap instruction. Compared to related technologies where encoding is required for every drawing instruction received, this reduces the number of encoding operations, lowers the consumption of encoding resources and time, thereby improving the smoothness and clarity of the remote desktop operation. Furthermore, the client decodes the encoded data to obtain decoded data. Because the relay server reduces the number of encoding operations, the client also reduces the number of decoding operations, lowering the client's consumption of decoding resources and time consumption, further improving the smoothness and clarity of the remote desktop operation. Additionally, the client draws images based on the decoded data and non-bitmap instructions. Since the non-bitmap instructions are not lost during transmission, this further ensures image clarity.

[0068] Figure 3 A schematic diagram of a data transmission system according to an embodiment of the present invention is shown. Figure 3 As shown, the data transmission system includes a remote desktop server 301, a relay server 302, and a client 303; among which,

[0069] The remote desktop server 301 is used to split the target image to be sent into multiple drawing instructions and send the multiple drawing instructions to the relay server, wherein the drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images.

[0070] The relay server 302 is used to draw a first image on a pre-created canvas according to the plurality of drawing instructions. During the process of drawing the first image, when a bitmap instruction is received, an encoding operation is triggered on the first drawn image according to the bitmap instruction to obtain encoded data. When a non-bitmap instruction is received, no encoding operation is performed on the second drawn image, and the encoded data and the non-bitmap instruction are sent to the client. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction.

[0071] The client 303 is used to decode the encoded data to obtain decoded data, and to draw an image based on the decoded data and the non-bitmap instructions.

[0072] In one embodiment, the relay server 302 is further configured to: draw a first image on the canvas sequentially according to the received drawing instructions in a linear order; during the drawing of the first image, detect whether the currently received drawing instruction is a bitmap instruction; if the currently received drawing instruction is a bitmap instruction, draw according to the bitmap instruction, and after drawing, trigger an encoding operation on the first drawn image to encode the first drawn image and obtain encoded data; if the currently received drawing instruction is a non-bitmap instruction, draw according to the non-bitmap instruction, and after drawing, do not trigger an encoding operation on the second drawn image.

[0073] In one embodiment, the relay server 302 is further configured to send the encoded data and the non-bitmap instructions to the client in a linear order of receiving the plurality of drawing instructions, wherein the position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

[0074] In one embodiment, the non-bitmap instruction carries the coordinate position information of the non-bitmap image in the client's canvas;

[0075] The client 303 is further configured to: draw a second image on a canvas pre-drawn on the client according to the decoded data; and overlay the non-bitmap image onto the area of ​​the second image corresponding to the coordinate position information according to the coordinate position information carried in the non-bitmap instruction.

[0076] The data transmission system provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0077] It should be noted that the embodiments of the data transmission system in this specification and the embodiments of the data transmission method in this specification are based on the same inventive concept. Therefore, for specific implementation of the data transmission system embodiments, please refer to the corresponding implementation of the data transmission method embodiments mentioned above. Repeated details will not be repeated.

[0078] The components in the data transmission system of this application embodiment can be devices, or components, integrated circuits, or chips in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0079] The data transmission device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0080] Based on the same technical concept, embodiments of this application also provide an electronic device for performing the above-described data transmission method. Figure 4 This is a schematic diagram of the structure of an electronic device to implement various embodiments of this application. The electronic device can vary significantly due to differences in configuration or performance, and may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call a computer program stored in the memory 430 and executable on the processor 410 to perform the following steps:

[0081] The remote desktop server splits the target image to be sent into multiple drawing instructions and sends the multiple drawing instructions to the relay server. The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images.

[0082] The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing process, when a bitmap instruction is received, the server triggers an encoding operation on the first drawn image to obtain encoded data. When a non-bitmap instruction is received, the server does not perform an encoding operation on the second drawn image. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction.

[0083] The relay server sends the encoded data and the non-bitmap instructions to the client;

[0084] The client decodes the encoded data to obtain decoded data, and draws an image based on the decoded data and the non-bitmap instructions.

[0085] In one embodiment, the relay server draws a first image on a pre-created canvas according to the plurality of drawing instructions, including: the relay server draws the first image on the canvas sequentially according to the received drawing instructions in a linear order.

[0086] Upon receiving a bitmap instruction, an encoding operation is triggered on the first drawn image to obtain encoded data according to the bitmap instruction. Upon receiving a non-bitmap instruction, no encoding operation is performed on the second drawn image. This includes: detecting whether the currently received drawing instruction is a bitmap instruction; if the currently received drawing instruction is a bitmap instruction, drawing is performed according to the bitmap instruction, and after drawing, an encoding operation is triggered on the first drawn image to encode the first drawn image and obtain encoded data; if the currently received drawing instruction is a non-bitmap instruction, drawing is performed according to the non-bitmap instruction, and after drawing, no encoding operation is triggered on the second drawn image.

[0087] In one embodiment, the relay server sends the encoded data and the non-bitmap instructions to the client, comprising: the relay server sending the encoded data and the non-bitmap instructions to the client in a linear order of receiving the plurality of drawing instructions, wherein the position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

[0088] In one embodiment, the non-bitmap instruction carries coordinate position information of the non-bitmap image on the client's canvas; the client draws the image based on the decoded data and the non-bitmap instruction, including:

[0089] The client draws a second image on a pre-drawn canvas based on the decoded data; and overlays the non-bitmap image onto the area of ​​the second image corresponding to the coordinate position information according to the coordinate position information carried in the non-bitmap instruction.

[0090] The specific execution steps can be found in the various steps of the above data transmission method embodiments, and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0091] It should be noted that the electronic devices in the embodiments of this application include: servers, terminals, or other devices besides terminals.

[0092] The above electronic device structure does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or arrange them differently. For example, an input unit may include a Graphics Processing Unit (GPU) and a microphone, and a display unit may use a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar display panels. User input units include at least one of a touch panel and other input devices. A touch panel is also called a touchscreen. Other input devices may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be elaborated further here.

[0093] Memory can be used to store software programs and various data. Memory can primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area can store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, memory can include volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0094] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and applications, while the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor.

[0095] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described data transmission method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0096] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0097] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0098] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0099] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0101] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A data transmission method, characterized in that, include: The remote desktop server splits the target image to be sent into multiple drawing instructions and sends the multiple drawing instructions to the relay server. The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images. The non-bitmap instructions carry the coordinate position information of the non-bitmap image in the client's canvas. The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing process, when a bitmap instruction is received, the server triggers an encoding operation on the first drawn image to obtain encoded data. When a non-bitmap instruction is received, the server does not perform an encoding operation on the second drawn image. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction. The relay server sends the encoded data and the non-bitmap instructions to the client; The client decodes the encoded data to obtain decoded data, and draws a second image on a canvas pre-drawn by the client based on the decoded data; Based on the coordinate position information carried in the non-bitmap instruction, the non-bitmap image is superimposed on the area in the second image corresponding to the coordinate position information.

2. The data transmission method according to claim 1, characterized in that, The relay server draws a first image on a pre-created canvas according to the multiple drawing instructions, including: The relay server draws the first image on the canvas in a linear order according to the received drawing instructions; Upon receiving a bitmap instruction, an encoding operation is triggered on the first drawn image according to the bitmap instruction to obtain encoded data. Upon receiving a non-bitmap instruction, no encoding operation is performed on the second drawn image, including: Detect whether the currently received drawing instruction is the bitmap instruction; If the currently received drawing instruction is a bitmap instruction, then drawing is performed according to the bitmap instruction, and after drawing, an encoding operation is triggered on the first drawn image to encode the first drawn image and obtain encoded data. If the currently received drawing instruction is a non-bitmap instruction, then drawing is performed according to the non-bitmap instruction, and no encoding operation on the second drawn image is triggered after drawing.

3. The data transmission method according to claim 1, characterized in that, The relay server sends the encoded data and the non-bitmap instructions to the client, including: The relay server sends the encoded data and the non-bitmap instructions to the client in a linear order of receiving the multiple drawing instructions, wherein the position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

4. A data transmission system, characterized in that, This includes a remote desktop server, a relay server, and a client; among which, The remote desktop server is used to split the target image to be sent into multiple drawing instructions and send the multiple drawing instructions to the relay server. The drawing instructions include bitmap instructions carrying bitmap images and non-bitmap instructions carrying non-bitmap images. The non-bitmap instructions carry the coordinate position information of the non-bitmap image in the client's canvas. The relay server is used to draw a first image on a pre-created canvas according to the multiple drawing instructions. During the drawing of the first image, when a bitmap instruction is received, an encoding operation is triggered on the first drawn image according to the bitmap instruction to obtain encoded data. When a non-bitmap instruction is received, no encoding operation is performed on the second drawn image, and the encoded data and the non-bitmap instruction are sent to the client. The first drawn image is the image that has been drawn on the canvas after drawing according to the bitmap instruction, and the second drawn image is the image that has been drawn on the canvas after drawing according to the non-bitmap instruction. The client is used to decode the encoded data to obtain decoded data, and draw a second image on a canvas pre-drawn on the client according to the decoded data; according to the coordinate position information carried in the non-bitmap instruction, the non-bitmap image is superimposed on the area of ​​the second image corresponding to the coordinate position information.

5. The data transmission system according to claim 4, characterized in that, The relay server is also used to draw a first image on the canvas in a linear order according to the received drawing instructions; During the process of drawing the first image, it is detected whether the currently received drawing instruction is a bitmap instruction; if the currently received drawing instruction is a bitmap instruction, drawing is performed according to the bitmap instruction, and after drawing, an encoding operation is triggered on the first drawn image to encode the first drawn image and obtain encoded data; if the currently received drawing instruction is a non-bitmap instruction, drawing is performed according to the non-bitmap instruction, and after drawing, no encoding operation is triggered on the second drawn image.

6. The data transmission system according to claim 4, characterized in that, The relay server is further configured to send the encoded data and the non-bitmap instructions to the client in a linear order of receiving the plurality of drawing instructions, wherein the position of the encoded data in the linear order is the position of the bitmap instruction that triggered the encoding operation to obtain the encoded data in the linear order.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the data transmission method as described in any one of claims 1-3.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the data transmission method as described in any one of claims 1-3.

Images