Remote desktop screen frame transmission method, system, electronic device and storage medium

Abstract

The application discloses a remote desktop screen frame transmission method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a remote desktop screen image; performing difference comparison on the remote desktop screen image and preset image data to obtain a preset difference array; determining a target pixel point according to the preset difference array to obtain preset pixel data, including pixel index data and pixel value data; performing difference value processing on the pixel index data to obtain index difference value data; performing encoding and compression processing on the index difference value data and the pixel value data to obtain preset transmission data; transmitting the preset transmission data to a preset control end; and when it is determined that the preset control end receives the preset transmission data, performing decoding and restoration processing on the preset transmission data, performing rendering on obtained image restoration data to obtain a target screen frame image. The application can improve the transmission efficiency of the remote desktop image and alleviate the problem of remote desktop display lag. The application can be widely applied in the technical field of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a remote desktop screen frame transmission method, system, electronic device and storage medium. Background Art

[0002] In daily work and life, remote operation is often required, such as remotely controlling another computer device from a local computer. In the related art, when performing remote operation, the remote screen displayed has problems such as image freeze and unsmooth display.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a remote desktop screen frame transmission method, system, electronic device and storage medium, which can effectively improve the transmission efficiency of remote desktop images and effectively alleviate the problem of remote desktop display freeze.

[0005] To achieve the above-mentioned purpose, an embodiment of the present application provides a remote desktop screen frame transmission method in one aspect, the method comprising the following steps:

[0006] Get remote desktop screen image;

[0007] Comparing the remote desktop screen image with preset image data to obtain a preset difference array; wherein the preset image data includes a previous frame of image data of the remote desktop screen image;

[0008] Determine a target pixel point that meets a preset pixel condition according to the preset difference array, and obtain preset pixel data; wherein the preset pixel data includes pixel index data and pixel value data;

[0009] Performing difference processing on the pixel index data of each target pixel point to obtain index difference data;

[0010] Performing encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data;

[0011] Transmitting the preset transmission data to a preset control terminal;

[0012] When it is determined that the preset control end has received the preset transmission data, the preset transmission data is decoded and restored, and the image restoration data obtained by decoding and restoration is rendered to obtain a target screen frame image.

[0013] In some embodiments, the preset image data further includes a preset image array;

[0014] The step of comparing the remote desktop screen image with preset image data to obtain a preset difference array includes:

[0015] Constructing the preset image array according to the preset screen resolution; wherein the preset image array serves as the first frame image data;

[0016] When it is determined that the remote desktop screen image is the first screen image, a pixel XOR calculation is performed on the remote desktop screen image and the preset image array according to a preset pixel unit to obtain the preset difference array; or, when it is determined that the remote desktop screen image is not the first screen image, a pixel XOR calculation is performed on the remote desktop screen image and the previous frame image data according to the preset pixel unit to obtain the preset difference array; wherein, the first screen image includes the image obtained by the first execution of the step of obtaining the remote desktop screen image.

[0017] In some embodiments, determining a target pixel point that meets a preset pixel condition according to the preset difference array to obtain preset pixel data includes:

[0018] Traversing the preset difference array to determine whether the pixel value data of each image pixel point in the preset difference array is zero;

[0019] When it is determined that the pixel value data is not zero, the image pixel point corresponding to the pixel value data is determined to be the target pixel point, and the preset pixel data is obtained.

[0020] In some embodiments, performing difference processing on the pixel index data of each target pixel point to obtain index difference data includes:

[0021] Constructing a current index variable; wherein the initial value of the current index variable is 0;

[0022] Traversing the target pixel points in order to subtract the current index variable from the pixel index data of the current pixel point to obtain a preset index difference; wherein the current pixel point includes the target pixel point currently selected during the traversal process;

[0023] The pixel index data is used as the current index variable, and the step of traversing the target pixel points in sequence to subtract the current index variable from the pixel index data of the current pixel point to obtain a preset index difference is returned, until the target pixel points are traversed and the index difference data is obtained according to the preset index difference.

[0024] In some embodiments, encoding and compressing the index difference data and the pixel value data to obtain preset transmission data includes:

[0025] Constructing a preset coding array; wherein the number of items in the preset coding array corresponds to the number of target pixels;

[0026] The preset index difference and the corresponding pixel value data are stored in the preset coding array in sequence to obtain a target coding array;

[0027] The target encoding array is compressed using a preset compression algorithm to obtain the preset transmission data.

[0028] In some embodiments, decoding and restoring the preset transmission data to render the image restoration data obtained by decoding and restoring to obtain a target screen frame image includes:

[0029] Decompressing the preset transmission data to obtain preset decompressed data; wherein the preset decompressed data includes the preset index difference and the corresponding pixel value data;

[0030] Performing difference restoration on the preset index difference to obtain original index data, so as to construct the image restoration data according to the original index data and the pixel value data;

[0031] Mapping rendering is performed according to the image restoration data to obtain the target screen frame image.

[0032] In some embodiments, performing difference restoration on the preset index difference to obtain original index data, and constructing the image restoration data according to the original index data and the pixel value data, includes:

[0033] Build a preset restoration array based on the preset screen resolution;

[0034] Constructing a restoration index variable; wherein the initial value of the restoration index variable is 0;

[0035] Traversing the preset decompressed data in order, adding the currently selected preset index difference value to the restored index variable to obtain the original index value;

[0036] The original index value is used as the restoration index variable, and the step of traversing the preset decompressed data in sequence is returned, and the currently selected preset index difference is added to the restoration index variable to obtain the original index value, until the preset decompressed data is traversed, and the original index value and the corresponding pixel value data are stored in sequence in the preset restoration array to obtain the image restoration data.

[0037] To achieve the above objectives, another aspect of the present application provides a remote desktop screen frame transmission system, the system comprising:

[0038] The first module is used to obtain the remote desktop screen image;

[0039] A second module is configured to compare the remote desktop screen image with preset image data to obtain a preset difference array; wherein the preset image data includes a previous frame of image data of the remote desktop screen image;

[0040] A third module is configured to determine a target pixel point that meets a preset pixel condition based on the preset difference array, and obtain preset pixel data; wherein the preset pixel data includes pixel index data and pixel value data;

[0041] A fourth module is configured to perform difference processing on the pixel index data of each target pixel point to obtain index difference data;

[0042] A fifth module is configured to perform encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data;

[0043] A sixth module, configured to transmit the preset transmission data to a preset control terminal;

[0044] The seventh module is used to decode and restore the preset transmission data when it is determined that the preset control end has received the preset transmission data, so as to render the image restoration data obtained by decoding and restoring, and obtain a target screen frame image.

[0045] To achieve the above-mentioned object, another aspect of the present application provides an electronic device, comprising:

[0046] at least one processor;

[0047] at least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0049] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned method when executed by a processor.

[0050] The embodiments of the present application include at least the following beneficial effects: The present application provides a remote desktop screen frame transmission method, system, electronic device, and storage medium. The scheme obtains a remote desktop screen image and compares the remote desktop screen image with preset image data to obtain a preset difference array. The preset image data includes the image data of the previous frame of the remote desktop screen image. Then, the embodiment of the present invention determines the target pixel points that meet the preset pixel conditions based on the preset difference array and obtains the corresponding preset pixel data, including pixel index data and pixel value data. Furthermore, the embodiment of the present invention performs difference processing on the pixel index data of each target pixel point to obtain index difference data, and encodes and compresses the index difference data and pixel value data to obtain preset transmission data, and transmits the preset transmission data to the preset control end. Finally, when it is determined that the preset control end has received the preset transmission data, the embodiment of the present invention decodes and restores the preset transmission data to render the image restoration data obtained by decoding and restoration to obtain the target screen frame image, thereby realizing the transmission of the remote desktop image. It is easy to understand that the embodiments of the present invention can effectively improve the compression efficiency by comparing the remote desktop screen image with the preset image data and performing difference processing on the pixel index data of each target pixel point, thereby effectively improving the transmission efficiency of the remote desktop image and effectively alleviating the problem of remote desktop display freeze. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flowchart of a remote desktop screen frame transmission method provided by an embodiment of the present invention;

[0052] Figure 2 This is a diagram of the image acquisition result data format provided by an embodiment of the present invention;

[0053] Figure 3 This is a data diagram of the first valued item in the sparse array provided by an embodiment of the present invention;

[0054] Figure 4 This is a data diagram of the second valued item in the sparse array provided by an embodiment of the present invention;

[0055] Figure 5 Schematic diagram of the storage structure of the first valued item in the preset coding array provided by an embodiment of the present invention;

[0056] Figure 6 2 is a schematic diagram of the storage structure of the second valued item in the preset coding array provided by an embodiment of the present invention;

[0057] Figure 7 This is a schematic diagram of byte stream data formed after encoding provided by an embodiment of the present invention;

[0058] Figure 8FIG. 1 is a schematic diagram of a position encoding process of a remote desktop screen frame transmission method according to an embodiment of the present application;

[0059] Figure 9 FIG. 2 is a schematic diagram of a remote desktop screen frame transmission system according to an embodiment of the present application;

[0060] Figure 10 FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0062] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0063] The terms "at least one", "multiple", "each", "any", and the like used in the present application include one, two or more, multiple includes two or more, and each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0065] Before the embodiments of the present application are described in detail, first, some nouns and terms involved in the embodiments of the present application are described, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0066] Frame image: The video stream is transmitted in the form of an image stream, where each image is a frame image.

[0067] Microsoft DirectX Graphics Infrastructure (DXGI): Provides underlying hardware device interface support for graphics libraries, such as enumerating graphics cards, setting up swap chains, and configuring front and back buffers. Not only D3D, but also libraries like GDI, OpenGL, and D2D require access to this interface for low-level hardware access. Its existence also enables interoperability between different graphics libraries. In DXGI, the resource interface is IDXGIResource, which can be converted into the interface of higher-level graphics libraries, such as D3D textures.

[0068] In daily work and life, remote operation is often required, such as remotely controlling another computer device from a local computer. In the related art, when performing remote operation, the remote screen displayed has problems such as image freeze and unsmooth display.

[0069] In view of this, the embodiments of the present application provide a remote desktop screen frame transmission method, system, electronic device and storage medium. The scheme obtains a remote desktop screen image and compares the remote desktop screen image with the preset image data to obtain a preset difference array. The preset image data includes the previous frame image data of the remote desktop screen image. Then, the embodiment of the present invention determines the target pixel points that meet the preset pixel conditions based on the preset difference array, and obtains the corresponding preset pixel data, including pixel index data and pixel value data. Furthermore, the embodiment of the present invention performs difference processing on the pixel index data of each target pixel point to obtain index difference data, and encodes and compresses the index difference data and pixel value data to obtain preset transmission data, and transmits the preset transmission data to the preset control end. Finally, when it is determined that the preset control end has received the preset transmission data, the embodiment of the present invention decodes and restores the preset transmission data to render the image restoration data obtained by decoding and restoration to obtain the target screen frame image, thereby realizing the transmission of the remote desktop image, effectively improving the transmission efficiency of the remote desktop image, and effectively alleviating the problem of remote desktop display freeze.

[0070] The remote desktop screen frame transmission method provided in the embodiment of the present application relates to the field of image processing technology. The remote desktop screen frame transmission method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the remote desktop screen frame transmission method, etc., but is not limited to the above forms.

[0071] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0072] Figure 1 This is an optional flowchart of the remote desktop screen frame transmission method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S170.

[0073] Step S110: Acquire the remote desktop screen image.

[0074] Step S120: performing a difference comparison between the remote desktop screen image and the preset image data to obtain a preset difference array, wherein the preset image data includes the previous frame image data of the remote desktop screen image.

[0075] Step S130: Determine target pixel points that meet preset pixel conditions based on the preset difference array to obtain preset pixel data, wherein the preset pixel data includes pixel index data and pixel value data.

[0076] Step S140: performing difference processing on the pixel index data of each target pixel point to obtain index difference data.

[0077] Step S150: performing encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data.

[0078] Step S160: Transmitting the preset transmission data to the preset control terminal.

[0079] Step S170: When it is determined that the preset control end receives the preset transmission data, the preset transmission data is decoded and restored, and the image restoration data obtained by decoding and restoration is rendered to obtain a target screen frame image.

[0080] During the working process of this specific embodiment, the embodiment of the present invention first obtains the remote desktop screen image to compare the remote desktop screen image with the preset image data to obtain the preset difference data. Specifically, in the embodiment of the present invention, the remote desktop screen image refers to the screen frame image of the remote desktop. For example, in the embodiment of the present invention, after receiving the instruction, the remote computer starts to collect screen images at high speed based on the DXGI technology according to the specified collection frequency (20-60 frames per second). Among them, the embodiment of the present invention adopts full-screen and full-pixel collection during collection, collects the RGB value of each pixel, and stores it according to a one-dimensional matrix table of X*Y length (pixel red value (R value), pixel green value (G value), pixel blue value (B value)). The R, G, and B values ​​are three 8-bit unsigned integers, and the storage method is: [[r,g,b], [r,g,b], [r,g,b]…] (total length X*Y), such as Figure 2 As shown. Accordingly, in an embodiment of the present invention, the preset image data includes the previous frame image data of the currently acquired remote desktop screen image. At the same time, the embodiment of the present invention determines the pixel points at which there are differences between the remote desktop screen image and the previous frame image by performing a difference comparison between the remote desktop screen image and its previous frame image data, thereby obtaining a corresponding preset difference array. For example, in an embodiment of the present invention, the preset difference array includes the pixel values ​​and corresponding pixel indexes of the pixel points at which there are differences between the currently acquired remote desktop screen image and the previous frame image.

[0081] Then, the embodiment of the present application determines the target pixel points meeting the preset pixel condition according to the preset difference array to obtain preset pixel data. Specifically, the preset pixel data in the embodiment of the present application includes index data and pixel value data. In the embodiment of the present application, the pixel sequence data of the target pixel point is taken as the index data of the pixel point. For example, when the maximum resolution of the display is 4K (4096x2160, the maximum index value is 4096x2160=8847360. In addition, the preset pixel condition in the embodiment of the present application includes the pixel value condition of each image pixel point stored in the preset difference array, such as whether the pixel value of the image pixel point meets the preset threshold condition. The embodiment of the present application analyzes the pixel value of each image pixel point stored in the preset difference array to determine the target pixel point and obtain the corresponding pixel index data and pixel value data. Further, the embodiment of the present application performs difference processing on the pixel index data of each target pixel point to obtain index difference data, and then performs encoding and compression processing on the index difference data and the pixel index data to obtain preset transmission data. Specifically, after the target pixel points are selected, the pixel index data of each target pixel point is processed by difference to obtain the corresponding index difference data. The index difference data of the target pixel point also corresponds to the pixel value data and the original pixel index data. Therefore, the embodiment of the present application replaces the original pixel index data with the index difference data, and performs encoding and compression processing on the index difference data and the corresponding pixel value data to obtain the preset transmission data, and then transmits the preset transmission data to the preset control end. The preset control end refers to a control end for remotely controlling a remote desktop screen.

[0082] Finally, when it is determined that the preset control end receives the preset transmission data, the embodiment of the present application performs encoding and restoration processing on the preset transmission to render the image restoration data obtained by encoding and restoration to obtain a target screen frame image. Specifically, the decoding and restoration processing in the embodiment of the present application corresponds to the encoding and compression processing. When the preset control end receives the preset transmission data, the embodiment of the present application restores the received preset transmission data by the reverse operation of the encoding and compression processing, i.e., decoding and restoration processing, to obtain image restoration data, i.e., the pixel index data and the pixel value data of the pixel point to be updated. Correspondingly, the embodiment of the present application renders the image restoration data locally to obtain a target screen image, realizing remote desktop frame transmission. In the embodiment of the present application, the changed pixel points are first determined by comparing the front and rear two images, and then the pixel index data of the selected target pixel points is processed by difference to obtain the index difference data, so that the compression efficiency of the preset transmission data can be further improved, the image can be accurately restored, the encoded image information is small enough to be smoothly transmitted under a small bandwidth, and the transmission effect of the remote desktop image is effectively improved, and the remote desktop display lag problem is alleviated.

[0083] In some embodiments of the present invention, the preset image data further includes a preset image array. Accordingly, in embodiments of the present invention, performing a difference comparison between the remote desktop screen image and the preset image data to obtain a preset difference array includes, but is not limited to, the following steps:

[0084] A preset image array is constructed according to the preset screen resolution, wherein the preset image array serves as the first frame of image data.

[0085] When the remote desktop screen image is determined to be the first screen image, an XOR calculation is performed on the pixel values ​​of the remote desktop screen image and a preset image array in a preset pixel unit to obtain a preset difference array. Alternatively, when the remote desktop screen image is determined not to be the first screen image, an XOR calculation is performed on the pixel values ​​of the remote desktop screen image and a previous frame of image data in a preset pixel unit to obtain a preset difference array. The first screen image includes the image obtained by the first execution of the step of obtaining the remote desktop screen image.

[0086] In this specific embodiment, the embodiment of the present invention first constructs a preset image array according to the preset screen resolution. Specifically, the embodiment of the present invention sets the preset image array as the first frame image data, that is, the preset image array is used as the first frame image data of the remote desktop screen image to be subsequently processed. Exemplarily, under the operating system, according to the acquired screen resolution (such as the screen resolution is X*Y), the embodiment of the present invention generates a one-dimensional array of X*Y length and three 8s as unsigned integers 0 according to the screen resolution to obtain the preset image array. Then, the embodiment of the present invention determines whether the acquired remote desktop screen image is the first screen image. Among them, the first screen image in the embodiment of the present invention refers to the image obtained by executing the step of acquiring the remote desktop screen image for the first time, that is, the remote desktop screen image acquired for the first time. Accordingly, when it is determined that the remote desktop screen image is the first screen image, the embodiment of the present invention performs pixel value XOR calculation based on the remote desktop screen image and the preset image array according to the preset pixel unit to obtain a preset difference array. Specifically, when the remote desktop screen image obtained is the first screen image, that is, the remote desktop screen image obtained for the first time, the embodiment of the present invention performs pixel value XOR calculation on the remote desktop screen image and the preset image array according to each pixel to generate a sparse array, that is, the preset difference array. In addition, when it is determined that the remote desktop screen image is not the first screen image, the embodiment of the present invention performs pixel value XOR calculation on the remote desktop screen image and the previous frame image data according to the preset pixel unit to obtain the preset difference array. Specifically, when the remote desktop screen image is not the image obtained for the first time, the embodiment of the present invention performs pixel value XOR calculation on the currently obtained remote desktop screen image and the previous frame image, that is, the previous frame image data, according to each pixel to obtain the preset difference array. It is easy to understand that the embodiment of the present invention constructs an empty image array, that is, the preset image array, so that all image frames are processed in the same way, and there is no need to process the image of the first frame separately, which alleviates the problem that the first frame image needs to be processed separately during the difference calculation process and it is necessary to clearly know which is the first frame image.

[0087] In some embodiments of the present invention, determining a target pixel point that meets a preset pixel condition according to a preset difference array to obtain preset pixel data includes but is not limited to the following steps:

[0088] The preset difference array is traversed to determine whether the pixel value data of each image pixel point in the preset difference array is zero.

[0089] When it is determined that the pixel value data is not zero, the image pixel point corresponding to the pixel value data is determined to be the target pixel point, and the preset pixel data is obtained.

[0090] In this specific embodiment, the embodiment of the present invention traverses the preset difference array to determine whether the pixel value data of each image pixel point in the preset difference array is zero. When it is determined that the pixel value data is not zero, the image pixel point corresponding to the pixel value data is determined to be the target pixel point, and the preset pixel data is obtained. Specifically, after performing a difference comparison between the remote desktop screen image and the preset image data, such as subtracting the remote desktop screen image from the preset image data or performing an XOR calculation to obtain the preset difference array, the embodiment of the present invention traverses the preset difference array, and determines whether the pixel value of each image pixel point in the preset difference array is zero by sequentially scanning the preset difference array to find items whose values ​​are not zero. Accordingly, when it is determined that the pixel data is not zero, that is, when an item whose pixel value is not zero is found, the embodiment of the present invention determines the image pixel point corresponding to the pixel value data as the target pixel point, and obtains corresponding preset pixel data, such as pixel value data and pixel index data, based on the target pixel point. It is easy to understand that after comparing the differences between the two frames of images (such as XOR calculation), the embodiment of the present invention filters out the items in the preset difference array whose pixel value data is not 0 to remove the pixels that have not changed, and only transmits the relevant data of the target pixel points, thereby effectively reducing the amount of data transmission and effectively improving the compression efficiency and transmission effect.

[0091] In some embodiments of the present invention, performing difference processing on the pixel index data of each target pixel to obtain index difference data includes but is not limited to the following steps:

[0092] Construct the current index variable. The initial value of the current index variable is 0.

[0093] The target pixel points are sequentially traversed to obtain a preset index difference by subtracting the current index variable from the pixel index data of the current pixel point, wherein the current pixel point includes the target pixel point currently selected during the traversal process.

[0094] The pixel index data is used as the current index variable, and the target pixel points are traversed in order to subtract the current index variable from the pixel index data of the current pixel point to obtain the preset index difference, until the target pixel point is traversed and the index difference data is obtained according to the preset index difference.

[0095] In this specific embodiment, the embodiment of the present invention first constructs the current index variable, and traverses the target pixel points in order to subtract the current index variable from the pixel index data of the current pixel point to obtain a preset index difference. Specifically, the initial value of the current index variable in the embodiment of the present invention is set to 0. For example, the embodiment of the present invention sets the current index variable to a 32-bit unsigned integer curr_a, and sets curr_a=0. Then, the embodiment of the present invention sequentially scans the items whose pixel values ​​are not zero in the preset difference array, that is, traverses the target pixel points in order to subtract the current index variable from the pixel index data of the currently selected target pixel point (current pixel point) to obtain the preset index difference corresponding to the target pixel point. For example, Figure 5 As shown, when the preset difference array is scanned in sequence and the first item whose pixel value is not 0 (i.e., the first target pixel point) is found, the embodiment of the present invention subtracts the current index variable from the pixel index data of the target pixel point to obtain its corresponding preset index difference. Further, the embodiment of the present invention uses the pixel index data as the current index variable and returns to the step of traversing the target pixel points in sequence to randomly subtract the current index variable from the pixel index of the current pixel point to obtain the preset index difference, until all target pixels are traversed and the index difference data is obtained according to the preset index difference. Specifically, after subtracting the value of curr_a from the index value of the item whose pixel value is not 0, the embodiment of the present invention sets curr_a to be equal to the index value of the item whose value is not 0. Then, the embodiment of the present invention searches for the next item whose pixel value is not 0 in sequence, i.e., the next target pixel point, to subtract the updated current index variable from the pixel index data of the next target pixel point, which is equivalent to subtracting the pixel index data of the previous target pixel point to obtain the corresponding preset index difference. Correspondingly, after traversing each target pixel point, the embodiment of the present invention obtains index difference data, that is, preset index difference values ​​of each target pixel point.

[0096] For example, Figure 3 、 Figure 4 、 Figure 5 as well as Figure 6 As shown, Figure 3 This is a data schematic diagram of the first valued item in the sparse array (preset difference array) provided by the embodiment of the present invention, that is, a data schematic diagram of the first target pixel point, and the index value of this target pixel point is 3197. At the same time, this is a data schematic diagram of the second valued item in the sparse array provided by the embodiment of the present invention, that is, a data schematic diagram of the second target pixel point, and the index value of this target pixel point is 6293. Therefore, when traversing the target pixel points in the sparse array (preset difference array) in sequence, when traversing to the first target pixel point, the index value of the first target pixel point is subtracted from the current index variable to obtain the preset index difference, that is, 3197-0=3197, as shown in FIG. Figure 5At the same time, the embodiment of the present invention sets the current index variable to 3197. Then, when traversing to the second target pixel point, the embodiment of the present invention subtracts the current index variable from the index value of the second target pixel point to obtain a preset index difference, that is, 6293-3197=3096, as shown in FIG. Figure 6 Accordingly, the embodiment of the present invention sets the current index variable to 6293, and continues to traverse and search for the next target pixel point until all target pixels are traversed.

[0097] In some embodiments of the present invention, encoding and compressing the index difference data and the pixel value data to obtain the preset transmission data includes but is not limited to the following steps:

[0098] Construct a preset encoding array, wherein the number of items in the preset encoding array corresponds to the number of target pixels.

[0099] The preset index difference and the corresponding pixel value data are stored in a preset coding array in sequence to obtain a target coding array.

[0100] The target encoding array is compressed using a preset compression algorithm to obtain preset transmission data.

[0101] In this specific embodiment, the embodiment of the present invention constructs a preset coding array to store the preset index difference and the corresponding pixel value components in the preset coding array in order to obtain a target coding array, and then compresses the target coding array through a preset compression algorithm to obtain preset transmission data. Specifically, the embodiment of the present invention determines the number of items in the preset coding array by calculating the number of non-zero value items in the preset difference array, that is, the number of target pixels, thereby constructing the preset coding array. For example, when there are M non-zero value items in the preset difference array, the embodiment of the present invention generates an array of M items, each item of the array has 6 bytes (the first three bytes represent the index difference, and the last three represent the R, G, and B values). Then, when the embodiment of the present invention sequentially scans the items (target pixels) in the preset difference array whose values ​​are not 0, the preset index difference and pixel value data of each target pixel point calculated are stored in the preset coding array in order, thereby constructing the target coding array. For example, the embodiment of the present invention stores the preset index difference and pixel value data of the first target pixel point in the first item of the preset coding array, and stores the preset index difference and pixel data of the second target pixel point in the second item of the preset coding array, and the subsequent other items are also stored in sequence, thereby obtaining a target coding array, such as Figure 7As shown. Finally, the embodiment of the present invention compresses the obtained target code array, for example, compresses the target code array through the ZSTD compression algorithm to obtain preset transmission data. It is easy to understand that in the compression algorithm of numerical values, for numerical values ​​of the same byte length, the more 0s the numerical value contains, the higher the compression efficiency. For different groups of random numerical values ​​of the same byte size (each group contains the same number of numerical values), from a probability perspective, the smaller the numerical value group, the more 0s it contains. Therefore, for the index value, the smaller the numerical value, the more 0s it contains, and the higher the compression efficiency. The embodiment of the present invention first calculates the index difference and encodes the index difference, such as using 3 bytes to encode the index difference, thereby improving the compression efficiency. From the use effect point of view, the compression rate is improved by about 8%, which can effectively improve the transmission effect and efficiency.

[0102] For example, when the three byte variables storing the index difference are a, b, and c, the pseudo code for storing the index difference in an embodiment of the present invention is as follows:

[0103]

[0104] Accordingly, the entire traversal comparison encoding pseudo code is as follows:

[0105]

[0106]

[0107] In some embodiments of the present invention, decoding and restoring the preset transmission data to render the image restoration data obtained by decoding and restoring to obtain the target screen frame image includes but is not limited to the following steps:

[0108] Decompressing the preset transmission data to obtain preset decompressed data, wherein the preset decompressed data includes preset index differences and corresponding pixel value data.

[0109] The preset index difference is restored to obtain the original index data, so as to construct the image restoration data according to the original index data and the pixel value data.

[0110] Mapping rendering is performed according to the image restoration data to obtain the target screen frame image.

[0111] In this specific embodiment, after receiving the preset transmission data at the preset control end, the embodiment of the present invention first decompresses the preset transmission data to obtain preset decompressed data. The preset index differences in the preset decompressed data are then differentially restored to obtain the original index data. Image restoration data is then constructed based on the original index data and the corresponding pixel value data. Mapping and rendering are then performed based on the image restoration data to obtain the target screen frame image. Specifically, the embodiment of the present invention first decompresses the byte stream sent from the remote end, i.e., the preset transmission data, to obtain the preset decompressed data. For example, the embodiment of the present invention decompresses the preset transmission data using a tool or library that supports the ZSTD format to obtain the preset decompressed data. Accordingly, the preset decompressed data obtained by decompression in the embodiment of the present invention also includes the preset index differences and the corresponding pixel value data. Next, because the index values ​​of each target pixel are differentially processed during the encoding process, the preset index differences need to be differentially restored to obtain the original index data. In the embodiment of the present invention, the original index data refers to the actual index value of each target pixel. Accordingly, the embodiment of the present invention constructs the current frame image data, i.e., the image restoration data, based on the restored original index data and the corresponding pixel value data. Finally, the embodiment of the present invention performs mapping according to the size of the display window and renders the image to the window to obtain the target screen frame image.

[0112] In some embodiments of the present invention, performing difference restoration on the preset index difference to obtain original index data, and constructing image restoration data based on the original index data and pixel value data, includes but is not limited to the following steps:

[0113] Builds an array of preset restores based on preset screen resolutions.

[0114] Construct a restore index variable, where the initial value of the restore index variable is 0.

[0115] Traverse the preset decompressed data in order, add the currently selected preset index difference to the restored index variable, and get the original index value.

[0116] The original index value is used as the restoration index variable, and the preset decompressed data is traversed in sequence. The currently selected preset index difference is added to the restoration index variable to obtain the original index value. Until the preset decompressed data is traversed, the original index value and the corresponding pixel value data are stored in the preset restoration array in sequence to obtain the image restoration data.

[0117] In this specific embodiment, the embodiment of the present invention first constructs a preset restoration array according to the preset screen resolution, and constructs a restoration index variable, and then traverses the preset decompressed data in order to add the preset index difference selected by the retaining ring to the restoration index variable to obtain the original index value, and at the same time uses the original index value as the restoration index variable, and returns to the step of traversing the preset decompressed data in order, adding the currently selected preset index difference to the restoration index variable to obtain the original index, until the traversal of the preset decompressed data is completed, and the original index value and the corresponding pixel value data are stored in the preset restoration array in order to obtain the image restoration data. Specifically, the embodiment of the present invention first generates a one-dimensional array of X*Y length and three 8-bit unsigned integers 0 locally according to the preset screen resolution (such as the resolution is X*Y), thereby constructing a preset restoration array. Accordingly, the embodiment of the present invention defines a restoration index variable, such as curr_b, and initializes the restoration index variable to zero, that is, curr_b=0. Next, the embodiment of the present invention reads data from the beginning of the decompressed byte stream (preset decompressed data), such as reading 6 bytes, and adds the preset index difference read to the restoration index variable to obtain the original index value. For example, the embodiment of the present invention uses the value of the first three bytes of the read 6-byte data + curr_b as the index of the first item of data in the preset restoration array, and stores the value of the last 3 bytes into the preset restoration array according to the index position, and updates the value of curr_b to the value of the first three bytes of the read 6-byte data + curr_b, that is, the restored original index value. Exemplarily, in the process of calculating the index difference, the embodiment of the present invention first reads the first three bytes into three 32-bit unsigned variables a, b, and c, then the index difference = a<<16+b<<8+c. Among them, the index value = index difference + curr_b. Then, the embodiment of the present invention sets curr_b = index value. Accordingly, the embodiment of the present invention constructs the image restoration data by repeating the above steps until all the original index values ​​are restored.

[0118] The following describes the solution of the embodiment of the present invention in detail with reference to a specific application example of remote desktop screen frame transmission:

[0119] For example, Figure 8 As shown, Figure 8A schematic diagram of the position encoding process of a remote desktop screen frame transmission method provided by an embodiment of the present invention. Specifically, the embodiment of the present invention first generates a one-dimensional array of three 8-bit unsigned integers (0) with a length of X*Y based on the screen resolution (e.g., the resolution is X*Y), i.e., a preset image array. Simultaneously, the embodiment of the present invention defines an empty array of six bytes with a length of X*Y as a sparse array for storing XORed values, i.e., a preset difference array. Then, the embodiment of the present invention captures the screen image. For example, the embodiment of the present invention connects to a remote computer. Upon receiving a command, the remote computer begins high-speed capture of the screen image based on a specified acquisition frequency (20-60 frames per second), using full-screen, full-pixel capture. The RGB values ​​of each pixel are collected and stored in a one-dimensional matrix table of length X*Y (pixel red value (R value), pixel green value (G value), and pixel blue value (B value)). The R, G, and B values ​​are three 8-bit unsigned integers. Each captured image frame is sent to the encoding and compression subsystem for encoding and compression. Accordingly, the embodiment of the present invention performs an XOR calculation on the pixel values ​​of the currently captured image and the previous frame of image data (if it is the first capture, it is XORed with the preset image array) for each pixel, and stores the calculation results in the sparse array constructed previously to obtain a preset difference array. Then, the embodiment of the present invention traverses the obtained sparse array (preset difference array) and begins encoding. Among them, since the current maximum display resolution is 4K (4096×2160, the maximum index value is 4096×2160=8847360), which can be represented by 24 bits, and the minimum encoding unit of the computer is one byte (8 bits), the embodiment of the present invention uses three bytes to store the index value. Next, the embodiment of the present invention calculates the number of non-zero value items in the sparse array generated in the previous step. If there are M non-zero value items, an array of M items is generated, with each item in the array having 6 bytes (the first three bytes represent the index difference, and the last three represent the R, G, and B values) as the preset encoding array. At the same time, the embodiment of the present invention defines the current index value variable 32-bit unsigned integer curr_a, and sets curr_a = 0. Then, the sparse array is scanned in sequence to determine the items in the sparse array whose pixel values ​​(RGB values) are not 0, the index value of the found item is subtracted by curr_a, and then curr_a = the index value of the currently found item whose pixel value is not 0, and the index difference calculated in the previous step and the pixel value (R, G, B) of the item whose pixel value is not 0 are stored in the first item of the preset coding array. Then, the embodiment of the present invention returns to the step of determining whether the sparse array has the next data, so as to continue to execute the step of traversing and scanning the sparse array until the scanning of the sparse array is completed. Finally, the embodiment of the present invention compresses the preset coding array that stores the index difference data and the pixel value data after pixel XOR calculation, and transmits it to the preset control end.Accordingly, when the preset control terminal receives the preset compressed data, it decodes and restores the preset transmission data to render the image restoration data obtained by decoding and restoring, thereby obtaining the target screen frame image.

[0120] It should be noted that the remote desktop frame transmission method of the embodiment of the present invention can be applied to desktop sharing or lossless video transmission. For example, computer B that provides desktop sharing, after taking screenshots, encoding, and compressing according to the above method, sends the data to end A, and computer A decompresses and decodes according to the above method to restore the image. Alternatively, in lossless video transmission, the transmitting end takes screenshots, encodes, and compresses according to the above method, and then sends the data to the receiving end, and the receiving end decompresses and decodes according to the above method to restore the image. For example, in daily work, it is used as a secure remote desktop tool to facilitate remote control of computers, that is, computer A controls computer B. The embodiment of the present invention deploys a TCP server in the intranet of the bastion host to realize the forwarding of TCP data. Computer B that needs to be remotely managed starts the client, and after setting the acquisition frequency of screen capture, starts and connects to the TCP server, generates a 32-bit login key and computer name, and then waits for the control end to access. Accordingly, remote computer A launches a client. After logging into the bastion host and passing authentication, the client enters the name and key of the computer to be controlled. The TCP server forwards the request to the controlled computer B. After confirming the key is passed, the controlling computer A opens the control window. The controlled computer B begins capturing screen images at a specified frequency, encodes and compresses them, and then sends them to the TCP server, which then forwards them to the controlling computer. Upon receiving the images, the controlling computer decompresses and decodes them, mapping the images according to the size of the local control window before rendering. In this embodiment of the present invention, the image capture and encoding of the remote computer and decoding of the remote computer A include a DXGI-based screenshot system, an encoding and compression system, and an encoding and compression restoration system. Accordingly, the screenshot system subsystem can capture 20-60 frames per second of remote desktop screen images. The encoding and compression subsystem encodes and compresses the captured images for transmission. The encoding and compression restoration subsystem is used to restore the received compressed images. Furthermore, upon receiving instructions, the screenshot system subsystem uses DXGI technology to perform high-speed screenshots, with a configurable rate of 20-60 screenshots per second. At the same time, the encoding and compression system performs data XOR processing on the intercepted image, calculates the index difference, encodes it, and then compresses it for transmission. Finally, the encoding and compression restoration system restores the image after receiving the encoded and compressed byte stream.

[0121] Accordingly, the embodiment of the present invention transmits only the portion that differs from the previous frame and employs a unique encoding technique, making the encoded difference portion very small. Furthermore, the non-zero value indexes of the difference sparse matrix are also differentially transmitted after compression and transmission, further reducing the amount of data transmitted. Furthermore, because the difference between each captured image and the previous frame is very small under normal circumstances due to high-speed acquisition, the embodiment of the present invention is not sufficient to encode and compress only the XOR value portion. Furthermore, the index of the difference portion is differentially encoded (since, for compression technology, smaller values ​​result in better compression effects). Upon receipt, the other party decompresses the image and calculates the true index based on the difference index. The difference pixel is then restored to the corresponding pixel point in the previous image based on the index to form the current image. The embodiment of the present invention is based on a high-speed screen image capture process, in which the difference between each frame and the previous frame is extremely small (in total observation, the difference is basically less than 3%. Assuming that it takes 2 seconds to open a window and display it, the window opening speed is captured at a speed of 20 frames per second, which is 2.5% per frame, meaning that the change in each frame is also 2.5%). Therefore, the first frame is automatically generated by the system according to the resolution and the value is all zero (not transmitted, not rendered, rendering starts from the second frame), and each subsequent frame only encodes and compresses the difference part for transmission. After the other party automatically generates an image with all values ​​0 after the first frame, each subsequent frame only restores the coordinate index of the received data and performs an XOR operation to restore the image.

[0122] It is easy to understand that when using the previous 6-byte encoding, that is, a three-byte storage index, the compression efficiency is not high because the index value is large. In the numerical compression algorithm, for values ​​of the same byte length, the more 0s the value contains, the higher the compression efficiency. For different groups of random values ​​of the same byte size (each group contains the same number of values), from a probability perspective, the smaller the value, the more 0s the group contains. Therefore, for the index value, the smaller the value, the more 0s it contains, and the higher the compression efficiency. Therefore, after using the index difference, the number of 0s is increased. For example, for 4K resolution, the maximum index value is 4096*2160-1=8837359, and after using the difference, the maximum difference is generally only 30% of the index value, which improves the compression efficiency. In actual tests, it was found that the compression efficiency can be improved by about 8%. At the same time, the use of pixel differences necessitates separate coding for the first frame, increasing the code workload. When processing a frame image, the amount of code added is almost doubled. Furthermore, since the first frame is processed separately, each code modification involves the addition of code, increasing the number of modifications and the probability of failure. This embodiment of the present invention uses an exclusive-OR operation to generate an empty image array at the beginning of processing. All image frames are processed identically, eliminating the need to process the first frame separately, effectively improving the reliability of remote desktop frame transmission.

[0123] See also Figure 9 The present application also provides a remote desktop screen frame transmission system that can implement the above-mentioned remote desktop screen frame transmission method. The system includes:

[0124] The first module 210 is configured to obtain a remote desktop screen image.

[0125] The second module 220 is configured to compare the remote desktop screen image with the preset image data to obtain a preset difference array, wherein the preset image data includes the previous frame of image data of the remote desktop screen image.

[0126] The third module 230 is configured to determine a target pixel point that meets a preset pixel condition according to a preset difference array, and obtain preset pixel data, wherein the preset pixel data includes pixel index data and pixel value data.

[0127] The fourth module 240 is configured to perform difference processing on the pixel index data of each target pixel point to obtain index difference data.

[0128] The fifth module 250 is configured to perform encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data.

[0129] The sixth module 260 is configured to transmit the preset transmission data to the preset control terminal.

[0130] The seventh module 270 is configured to decode and restore the preset transmission data when it is determined that the preset control terminal has received the preset transmission data, and render the image restoration data obtained by the decoding and restoration to obtain a target screen frame image.

[0131] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0132] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the remote desktop screen frame transmission method. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0133] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] See also Figure 10 , Figure 10The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0135] The processor 310 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0136] The memory 320 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 320 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 320 and is called by the processor 310 to execute the remote desktop screen frame transmission method of the embodiments of this application.

[0137] Input / output interface 330, used to implement information input and output;

[0138] Communication interface 340, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0139] bus 350 , which transmits information between the various components of the device (e.g., processor 310 , memory 320 , input / output interface 330 , and communication interface 340 );

[0140] The processor 310 , the memory 320 , the input / output interface 330 and the communication interface 340 are connected to each other in communication within the device via the bus 350 .

[0141] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned remote desktop screen frame transmission method.

[0142] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0143] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0145] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0146] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0147] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0148] The terms "first", "second", "third", "fourth" and the like used in the specification of the present application and the above-described drawings, if any, are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0149] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0150] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0151] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0152] In addition, the functional units in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0154] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A remote desktop screen frame transmission method, characterized in that: The method comprises the following steps: Get remote desktop screen image; Comparing the remote desktop screen image with preset image data to obtain a preset difference array; wherein the preset image data includes a previous frame of image data of the remote desktop screen image; Determine a target pixel point that meets a preset pixel condition according to the preset difference array, and obtain preset pixel data; wherein the preset pixel data includes pixel index data and pixel value data; Performing difference processing on the pixel index data of each target pixel point to obtain index difference data; Performing encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data; Transmitting the preset transmission data to a preset control terminal; When it is determined that the preset control end has received the preset transmission data, the preset transmission data is decoded and restored, and the image restoration data obtained by decoding and restoration is rendered to obtain a target screen frame image.

2. The method according to claim 1, characterized in that The preset image data also includes a preset image array; The step of comparing the remote desktop screen image with preset image data to obtain a preset difference array includes: Constructing the preset image array according to the preset screen resolution; wherein the preset image array serves as the first frame image data; When it is determined that the remote desktop screen image is the first screen image, a pixel XOR calculation is performed on the remote desktop screen image and the preset image array according to a preset pixel unit to obtain the preset difference array; or, when it is determined that the remote desktop screen image is not the first screen image, a pixel XOR calculation is performed on the remote desktop screen image and the previous frame image data according to the preset pixel unit to obtain the preset difference array; wherein, the first screen image includes the image obtained by the first execution of the step of obtaining the remote desktop screen image.

3. The method according to claim 1, characterized in that The step of determining a target pixel point that meets a preset pixel condition according to the preset difference array to obtain preset pixel data includes: Traversing the preset difference array to determine whether the pixel value data of each image pixel point in the preset difference array is zero; When it is determined that the pixel value data is not zero, the image pixel point corresponding to the pixel value data is determined to be the target pixel point, and the preset pixel data is obtained.

4. The method according to claim 3, characterized in that The performing difference processing on the pixel index data of each target pixel point to obtain index difference data includes: Constructing a current index variable; wherein the initial value of the current index variable is 0; Traversing the target pixel points in order to subtract the current index variable from the pixel index data of the current pixel point to obtain a preset index difference; wherein the current pixel point includes the target pixel point currently selected during the traversal process; The pixel index data is used as the current index variable, and the step of traversing the target pixel points in sequence to subtract the current index variable from the pixel index data of the current pixel point to obtain a preset index difference is returned, until the target pixel points are traversed and the index difference data is obtained according to the preset index difference.

5. The method according to claim 4, characterized in that The encoding and compressing of the index difference data and the pixel value data to obtain preset transmission data includes: Constructing a preset coding array; wherein the number of items in the preset coding array corresponds to the number of target pixels; The preset index difference and the corresponding pixel value data are stored in the preset coding array in sequence to obtain a target coding array; The target encoding array is compressed using a preset compression algorithm to obtain the preset transmission data.

6. The method according to claim 4, characterized in that The decoding and restoring processing of the preset transmission data to render the image restoration data obtained by decoding and restoring to obtain the target screen frame image includes: Decompressing the preset transmission data to obtain preset decompressed data; wherein the preset decompressed data includes the preset index difference and the corresponding pixel value data; Performing difference restoration on the preset index difference to obtain original index data, so as to construct the image restoration data according to the original index data and the pixel value data; Mapping rendering is performed according to the image restoration data to obtain the target screen frame image.

7. The method according to claim 6, characterized in that The performing difference restoration on the preset index difference to obtain original index data, and constructing the image restoration data according to the original index data and the pixel value data, includes: Build a preset restoration array based on the preset screen resolution; Constructing a restoration index variable; wherein the initial value of the restoration index variable is 0; Traversing the preset decompressed data in order, adding the currently selected preset index difference value to the restored index variable to obtain the original index value; The original index value is used as the restoration index variable, and the step of traversing the preset decompressed data in sequence is returned, and the currently selected preset index difference is added to the restoration index variable to obtain the original index value, until the preset decompressed data is traversed, and the original index value and the corresponding pixel value data are stored in sequence in the preset restoration array to obtain the image restoration data.

8. A remote desktop screen frame transmission system, characterized in that: The system comprises: The first module is used to obtain the remote desktop screen image; A second module is configured to compare the remote desktop screen image with preset image data to obtain a preset difference array; wherein the preset image data includes a previous frame of image data of the remote desktop screen image; A third module is configured to determine a target pixel point that meets a preset pixel condition based on the preset difference array, and obtain preset pixel data; wherein the preset pixel data includes pixel index data and pixel value data; A fourth module is configured to perform difference processing on the pixel index data of each target pixel point to obtain index difference data; A fifth module is configured to perform encoding and compression processing on the index difference data and the pixel value data to obtain preset transmission data; A sixth module, configured to transmit the preset transmission data to a preset control terminal; The seventh module is used to decode and restore the preset transmission data when it is determined that the preset control end has received the preset transmission data, so as to render the image restoration data obtained by decoding and restoring, and obtain a target screen frame image.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images