Remote desktop frame query method and device

Abstract

The invention discloses a remote desktop frame query method and device, and relates to the field of remote desktops, and the method comprises the steps: dividing a current frame into a plurality of target bitmap lines corresponding to a displacement direction according to the pixel length of the bitmap lines, and calculating the hash values of the target bitmap lines; the Hash values are stored in a target two-dimensional array, the Hash values located in the same row form a continuous storage dimension, and the Hash values located in different rows form a discrete storage dimension, or the Hash values located in the same column form the continuous storage dimension, and the Hash values located in different columns form the discrete storage dimension; aiming at each target hash value in the target two-dimensional array, inquiring and judging whether a first hash value with the same row and column sequence number as the target hash value in a first two-dimensional array of a previous frame is the same as the target hash value or not; and if the first hash value is the same as the second hash value, determining that the processing mode of the target bitmap line corresponding to the target hash value is moving, so that the client takes the data corresponding to the first hash value as the data of the target hash value, thereby effectively reducing bandwidth occupation, CPU consumption and image display delay.

Description

Technical Field

[0001] The present application relates to the field of remote desktop technology, and in particular to a remote desktop frame query method and device. Background Art

[0002] In the field of remote desktop, when the remote desktop software transmits the display image of desktop A to desktop B for display through the network, in order to reduce bandwidth usage, each frame of the desktop image is usually divided into multiple blocks, and the hash value of each block is calculated, and the hash value is stored in the cache hash table for fast search. Therefore, when a new frame of the desktop image is obtained, the cache hash table is searched for blocks with the same hash value. If no block with the same hash value as the block is found, the block is encoded and sent to the client; if a block with the same hash value as the block is found, there is no need to encode the block, and the client is directly notified to copy the data of the block from the locally cached blocks to a certain area for display.

[0003] However, since each block must be compared with all blocks of the previous n frames, there are problems such as high CPU (Central Processing Unit) occupancy and high desktop display delay, which in turn affects the user experience. Summary of the invention

[0004] The present application provides a remote desktop frame query method and device, which can solve the technical problems of high CPU occupancy and high desktop display delay in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a remote desktop frame query method, the remote desktop frame query method comprising:

[0006] Divide the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line;

[0007] Store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0008] For each target hash value in the target two-dimensional array, query and determine whether a first hash value in a first two-dimensional array corresponding to a previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value;

[0009] If they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0010] In combination with the first aspect, in one embodiment, when the displacement direction is horizontal displacement, hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension; when the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

[0011] In combination with the first aspect, in one embodiment, when the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, it also includes: storing the target hash value in the target two-dimensional array and the column number of the column in which it is located in the target hash table corresponding to the row where the target hash value is located in the target hash table array corresponding to the current frame in the form of key-value pairs.

[0012] In combination with the first aspect, in one implementation, after the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the method further includes:

[0013] If the first hash value is different from the target hash value, determining whether a processing mode of a bitmap line corresponding to a second hash value in the target two-dimensional array is movement, wherein the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value is located before the target hash value;

[0014] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, the third hash value and the target hash value having the same row number and a column number of the third hash value being determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0015] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0016] In combination with the first aspect, in one implementation, if the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes:

[0017] Determine whether there is a fourth hash value that is the same as the target hash value in the target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0018] If so, determine the first column number according to the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is movement, and the displacement value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0019] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0020] In combination with the first aspect, in one implementation, the method further includes:

[0021] The target hash values ​​in the same row of the target two-dimensional array are grouped according to a preset grouping length to obtain a plurality of hash value groups and calculate a total hash value of each hash value group;

[0022] All total hash values ​​are stored in a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0023] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row where the target total hash value is located in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0024] In combination with the first aspect, in one implementation, the method further includes:

[0025] For each target total hash value in the target total two-dimensional array, when it is detected that there is a bitmap line whose processing method is encoding in the bitmap line corresponding to the target total hash value, it is determined whether there is a target frame in the previous N frames, and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner, and the first total hash value has the same row number as the target total hash value and the column number of the first total hash value is less than the column number of the target total hash value, wherein N is a positive integer;

[0026] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0027] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as copy cache and the displacement value is the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0028] In combination with the first aspect, in one implementation, if there is no target frame in the first N frames, all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache or the second total hash value is different from the target total hash value, the method further includes:

[0029] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0030] If so, determine the second column number according to the key-value pair corresponding to the third total hash value, and determine that the processing method of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column number and the column number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0031] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0032] In combination with the first aspect, in one implementation, before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line, the method further includes:

[0033] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0034] If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0035] If not, the current frame is encoded in streaming mode.

[0036] In a second aspect, an embodiment of the present application provides a remote desktop frame query device, the remote desktop frame query device comprising:

[0037] A displacement identification module, which is used to divide the current frame into a plurality of target bitmap lines corresponding to the displacement direction thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0038] A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0039] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0040] The current frame is divided into a plurality of target bitmap lines corresponding to the displacement directions according to a preset bitmap line pixel length and a hash value of each target bitmap line is calculated; all hash values ​​are stored in a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; for each target hash value in the target two-dimensional array, query and determine whether the first hash value having the same row and column sequence as the target hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value; if the same, determine that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value to realize data query of the current frame. When comparing the current frame with the previous frame, the present application uses a bitmap line as the smallest unit of cache query. Compared with the traditional image block solution, the comparison area is smaller and has a higher hit rate, so that the occupied network bandwidth will be smaller; and because the bitmap line of the current frame will only be compared with the bitmap line of the previous frame with the same discrete storage dimension, the number of comparisons is less than the image block solution that requires comparison with all blocks of the previous n frames, so that the CPU consumption is smaller and the image display delay is lower. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flowchart of an embodiment of the remote desktop frame query method of the present application;

[0042] Figure 2 This is a schematic diagram of a horizontal bit shift line involved in the embodiment of the present application;

[0043] Figure 3 A schematic diagram of a vertical displacement line diagram involved in an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the displacement matching process involved in the embodiment of the present application;

[0045] Figure 5 A schematic diagram of a horizontal bit shift line group involved in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of a vertically shifted bitmap line group involved in an embodiment of the present application;

[0047] Figure 7 A schematic diagram of a cache matching process involved in an embodiment of the present application;

[0048] Figure 8 This is a schematic diagram of the canvas involved in the embodiment scheme of the present application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0051] In a first aspect, an embodiment of the present application provides a remote desktop frame query method.

[0052] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the remote desktop frame query method of the present application. Figure 1 As shown, the remote desktop frame query method includes:

[0053] Step S10: Divide the current frame into a plurality of target bitmap lines corresponding to the displacement direction thereof according to the preset bitmap line pixel length and calculate the hash value of each target bitmap line.

[0054] Exemplarily, in this embodiment, a bitmap line refers to a line composed of multiple pixels in a bitmap, wherein the pixel length of the bitmap line can be determined according to actual needs and is not limited here. For example, if the pixel length of the bitmap line is preset to M, the number of pixels corresponding to each bitmap line is M, that is, M consecutive pixels constitute a bitmap line. In this embodiment, the displacement recognition module in the server will use a fast algorithm to compare the bitmap information of the current frame and the previous frame to estimate whether the current frame is horizontally displaced or vertically displaced relative to the previous frame, and divide the current frame into multiple horizontally displaced bitmap lines or vertically displaced bitmap lines according to the recognition result. If there is no displacement, the displacement direction of the current frame is determined according to the displacement direction of the previous frame. For example, if the displacement direction of the previous frame is horizontal displacement, the displacement direction of the current frame is also horizontal displacement, and if the displacement direction of the previous frame is vertical displacement, the displacement direction of the current frame is also vertical displacement.

[0055] For horizontal displacement, each column of pixels in the current frame will be divided according to the preset bitmap line pixel length M, so that each column is divided into multiple target bitmap lines, and the number of target bitmap lines in each row is the same as the number of pixels in a row; for example, the total number of pixels in each row of the current frame image is S and the total number of pixels in each column is P, then the number of target bitmap lines corresponding to a column is The number of target bitmap lines corresponding to one row is S, so the total number of target bitmap lines in the horizontal displacement direction of the current frame is

[0056] For vertical displacement, each row of pixels in the current frame will be divided according to the preset bitmap line pixel length M, so that each row is divided into multiple target bitmap lines, and the number of target bitmap lines in each column is the same as the number of pixels in a column; for example, the total number of pixels in each row of the current frame image is S and the total number of pixels in each column is P, then the number of target bitmap lines corresponding to one row is The number of target bitmap lines corresponding to a column is P, so the total number of target bitmap lines in the vertical displacement direction of the current frame is

[0057] After the bitmap line division is completed, the corresponding hash value will be calculated according to the RGBA (RGBA represents the color space of red, green, blue and alpha) value of each target bitmap line.

[0058] Furthermore, in one embodiment, before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement direction thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line, the method further includes:

[0059] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0060] If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0061] If not, the current frame is encoded in streaming mode.

[0062] For example, it is understandable that most high-hit rate desktop scenes have the characteristics of only horizontal displacement or vertical displacement (such as browsing web pages, documents, etc.), while a small number of high-hit rate desktop scenes have the characteristics of both horizontal displacement and vertical displacement (such as dragging windows and pictures, etc.); for a small number of high-hit rate desktop scenes and low-hit rate desktop scenes (such as playing videos), using cache query to reduce network bandwidth is a very cost-effective method. Therefore, in this embodiment, different frame query methods will be used for different cache hit rate scenes.

[0063] Specifically, the server in this embodiment also includes a bitmap capture module, a scene recognition module, a displacement matching module, a cache matching module, an encoding module, and a protocol sending module. Among them, the bitmap capture module captures the bitmap information of the desktop image every time the user's desktop image changes, and sends it to the scene recognition module; the scene recognition module will perform scene recognition based on the bitmap information, and if the current scene is a desktop scene with a high hit rate, the bitmap information of the desktop image will be sent to the displacement recognition module for cache query; and if the current scene is a desktop scene with a small number of high hit rates or a desktop scene with a low hit rate, the bitmap information of the desktop image will be sent to the encoding module, so that the encoding module directly performs encoding processing in a streaming mode and sends it to the client through the protocol sending module.

[0064] It should be noted that the desktop scene type of the current frame can be determined based on the desktop scene type of the previous frame. For example, if the previous frame is a preset desktop scene with a high hit rate, then the current frame is also a preset desktop scene with a high hit rate. If the previous frame is a desktop scene with a low hit rate, then the current frame is also a desktop scene with a low hit rate. In addition, during the frame query process of the current frame, the desktop scene type of the current frame will also be updated and calculated to provide a basis for the desktop scene type of the next frame. It should be understood that the calculation of desktop scenes with a high hit rate, a small number of desktop scenes with a high hit rate, and a desktop scene with a low hit rate is common knowledge in the art and will not be repeated here.

[0065] It can be seen that this embodiment mainly optimizes the bandwidth occupancy and CPU occupancy of most of the above-mentioned high-hit rate desktop scenarios. When a small number of high-hit rate desktop scenarios and low-hit rate desktop scenarios are identified, it can automatically switch to streaming mode to encode the desktop image in full screen, thereby reducing CPU occupancy and delay; and when most of the high-hit rate desktop scenarios are identified, it can automatically switch to cache mode to reduce bandwidth occupancy and CPU occupancy.

[0066] Step S20: All hash values ​​are stored in a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension.

[0067] Specifically, when the displacement direction is horizontal displacement, hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension;

[0068] When the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

[0069] Wherein, when the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, the method further includes:

[0070] The target hash value in the target two-dimensional array and the column number of the column where it is located are stored in the target hash table corresponding to the row where the target hash value is located in the target hash table array corresponding to the current frame in the form of key-value pairs.

[0071] Exemplarily, in this embodiment, a two-dimensional array and a hash table array are constructed for each frame through the displacement recognition module to store the hash value and the hash table, respectively. It should be noted that since the data storage method and principle of each frame are the same, for the sake of simplicity of description, the following embodiment will take the data storage of the current frame as an example to illustrate the assignment of the two-dimensional array and the hash table array.

[0072] Specifically, when assigning values ​​to a two-dimensional array, if the displacement direction of the current frame image is horizontal displacement, the hash values ​​located in the same row will be stored according to a continuous storage dimension, and the hash values ​​located in the same row of the current frame image will constitute a continuous storage dimension while the hash values ​​located in different rows will constitute a discrete storage dimension; if the displacement direction of the current frame image is vertical displacement, the hash values ​​located in the same column of the current frame image will be stored according to a continuous storage dimension, and the hash values ​​located in the same column will constitute a continuous storage dimension while the hash values ​​located in different columns will constitute a discrete storage dimension.

[0073] Assuming the desktop resolution is 1920×1080, the horizontal displacement corresponds to a bitmap line pixel length of 64, and the vertical displacement corresponds to a bitmap line pixel length of 128; see Figure 2 As shown, when the displacement recognition result of the current frame is horizontal displacement, the displacement recognition module will divide each column according to 64 pixel units, and the desktop image is divided into 17 rows and 1920 columns, wherein the length of the target bitmap line of the first 16 rows is 64, and the length of the target bitmap line of the 17th row is 56; then the hash values ​​of the 1920 target bitmap lines (i.e., horizontal displacement bitmap lines) corresponding to each row are stored in the corresponding row of the target two-dimensional array a according to the continuous storage dimension. For example, the hash values ​​of the first row in the current frame image are sequentially stored in the first row of the target two-dimensional array a. The hash values ​​of the second row are sequentially stored in the second row of the target two-dimensional array a, and so on. The hash values ​​on different rows of the target two-dimensional array a will constitute discrete storage dimensions, forming the target two-dimensional array a

[17] , that is, the target two-dimensional array a includes 17 rows and 1920 columns; it can be understood that the target bitmap line hash value corresponding to the yth row and xth column in the current frame is stored in the element a[y][x] of the yth row and xth column in the target two-dimensional array a

[17] , and for the target two-dimensional array a

[17] , the element of the yth row and xth column can also refer to the element with discrete dimension y and continuous dimension x.

[0074] Similarly, see Figure 3 As shown, when the displacement recognition result of the current frame is a vertical displacement, the displacement recognition module will divide each row according to 128 pixels, and the desktop image will be divided into 1080 rows and 15 columns, wherein the target bitmap line length of each column is 128; then the hash values ​​of the 1080 target bitmap lines (i.e., vertical displacement bitmap lines) corresponding to each column are stored in the corresponding rows of the target two-dimensional array a according to the continuous storage dimension, for example, the hash value of the first column in the current frame image is sequentially stored in the first row of the target two-dimensional array a, and the hash value of the second column in the current frame image is sequentially stored in the second row of the target two-dimensional array a, and so on, then the hash values ​​on different columns in the target two-dimensional array a will constitute a discrete storage dimension, thus forming a target two-dimensional array a

[15] , i.e., the target two-dimensional array a includes 15 rows and 1080 columns; it can be understood that the target bitmap line hash value corresponding to the yth row and xth column in the current frame is stored in the element a[x][y] of the xth row and yth column in the target two-dimensional array a

[15] .

[0075] After completing the assignment of the two-dimensional array, the hash table array will also be assigned. Since the hash table array assignment method and principle corresponding to the horizontal displacement and the vertical displacement are similar, for the sake of simplicity, the following will take the horizontal displacement as an example to explain the assignment of the hash table array: the row number in the target two-dimensional array a is used as the element index value of the target hash table array v, then the number of elements in the target hash table array v is consistent with the number of rows in the target two-dimensional array a; then according to the continuous storage dimension, the hash values ​​of each row in the target two-dimensional array a and the column number of the column to which it belongs are stored in the hash table at the location of the corresponding element index value in the target hash table array v in the form of key-value pairs; wherein, in each key-value pair, the hash value is used as the key and the column number is used as the value, and each element in the target hash table array v is a hash table. For example, assuming that the hash value corresponding to the element a[y][x] in the yth row and xth column of the target two-dimensional array a is h, then the (h, x) key-value pair is stored in the hash table v[y] corresponding to the yth element of the target hash table array v.

[0076] It should be noted that if the displacement direction of the current frame is different from that of the previous frame, the two-dimensional array and the hash table array of the previous frame need to be reconstructed according to the displacement direction of the current frame.

[0077] Step S30: for each target hash value in the target two-dimensional array, query and determine whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value.

[0078] Exemplarily, in this embodiment, a bitmap line processing mode table is first initialized through the displacement matching module in the server to record the processing mode to be performed on each target bitmap line of the current frame, and the default value is no processing; specifically, the target two-dimensional array of the current frame is matched with the previous frame in the order of continuous storage dimension first and then discrete storage dimension, and then the processing mode of each bitmap line is set according to the matching result.

[0079] Since the frame query method and principle of each target hash value in the target two-dimensional array are similar regardless of horizontal displacement or vertical displacement, for the sake of simplicity of description, the following embodiment will take one of the target hash values ​​under horizontal displacement as an example to illustrate the frame query of the corresponding target bitmap line. It is assumed that a frame query is required for the target bitmap line corresponding to the target hash value of the element a[y][x] with a discrete dimension of y and a continuous dimension of x in the target two-dimensional array a corresponding to the current frame: determine whether the value of the element la[y][x] with a discrete dimension of y and a continuous dimension of x in the first two-dimensional array la corresponding to the previous frame (i.e., the first hash value) is equal to the value of the element a[y][x] in the target two-dimensional array a (i.e., the target hash value), and then determine whether the frame query of the target bitmap line corresponding to the target hash value is completed based on the judgment result.

[0080] It can be seen that, in this embodiment, when comparing the current frame with the previous frame, the bitmap line is used as the smallest unit of cache query. Compared with the traditional image block solution, the comparison area is smaller and has a higher hit rate, so that the occupied network bandwidth will be smaller; and because the bitmap line of the current frame will only be compared with the bitmap line of the previous frame with the same discrete storage dimension, the number of comparisons is less than the image block solution that needs to compare with all the blocks of the previous n frames, so that the CPU consumption is smaller and the image display delay is lower.

[0081] Step S40: If they are the same, determining that the processing method of the target bitmap line corresponding to the target hash value is moving, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0082] Exemplarily, in this embodiment, if the first hash value (i.e., the value corresponding to la[y][x]) is equal to the target hash value (i.e., the value corresponding to a[y][x]), it means that the frame query of the target bitmap line corresponding to the target hash value has been completed, and the processing method of the target bitmap line corresponding to the target hash value is recorded as move, so that the client can copy the data corresponding to the first hash value according to the processing method, thereby completing the frame query of one bitmap line and continuing with the frame query processing of the next bitmap line.

[0083] Furthermore, in one embodiment, after the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the following step is further included:

[0084] If the first hash value is different from the target hash value, determining whether a processing mode of a bitmap line corresponding to a second hash value in the target two-dimensional array is movement, wherein the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value is located before the target hash value;

[0085] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, the third hash value and the target hash value having the same row number and a column number of the third hash value being determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0086] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0087] For example, see Figure 4As shown, if the first hash value (i.e., the value corresponding to la[y][x]) is not equal to the target hash value (i.e., the value corresponding to a[y][x]), it will continue to determine whether the bitmap line processing method corresponding to the value of the element a[y][x-1] with a discrete dimension of y and a continuous dimension of x-1 in the target two-dimensional array a corresponding to the current frame (i.e., the second hash value) is moving. If it is moving and the displacement value is o, it is determined whether the value of the element la[y][x+o] with a discrete dimension of y and a continuous dimension of x+o in the first two-dimensional array la corresponding to the previous frame (i.e., the third hash value) is equal to the value of a[y][x]. If they are equal, it is recorded that the processing method of the target bitmap line with a discrete dimension of y and a continuous dimension of x in the current frame is moving and the displacement value is o, so as to complete the frame query of the target bitmap line, so that the client can copy the data corresponding to the third hash value according to the processing method and the displacement value.

[0088] Furthermore, in one embodiment, if the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes:

[0089] Determine whether there is a fourth hash value that is the same as the target hash value in the target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0090] If so, determine the first column number according to the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is movement, and the displacement value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0091] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0092] For example, see Figure 4As shown, if the bitmap line processing method corresponding to the value of a[y][x-1] is not moving, or the value of la[y][x+o] is not equal to the value of a[y][x], the query will continue to check whether there is a key (i.e., the fourth hash value) equal to the value of a[y][x] in the y-th hash table lv[y] in the first hash table array lv corresponding to the previous frame. If there are equal keys and the continuous storage dimension index of the corresponding hash value is lx (i.e., the first column sequence number), then the processing method of the target bitmap line with a discrete dimension of y and a continuous dimension of x in the current frame is recorded as moving and the displacement value is lx-x, so as to complete the frame query of the target bitmap line, so that the client can copy the data corresponding to the fourth hash value according to the processing method; if there are no equal keys, then the processing method of the target bitmap line with a discrete dimension of y and a continuous dimension of x in the current frame is recorded as encoding.

[0093] Furthermore, in one embodiment, the method further includes:

[0094] The target hash values ​​in the same row of the target two-dimensional array are grouped according to a preset grouping length to obtain a plurality of hash value groups and calculate a total hash value of each hash value group;

[0095] All total hash values ​​are stored in a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0096] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row where the target total hash value is located in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0097] Exemplarily, it should be noted that the group length refers to the number of bitmap lines (i.e., hash values) contained in a bitmap line group (i.e., hash value group). For example, if the group length is g, then a bitmap line group contains g bitmap lines, that is, a hash value group contains g hash values. The specific value setting can be determined according to actual needs and is not limited here. In this embodiment, the cache matching module groups the bitmap lines according to the group length g in the continuous dimension to form a bitmap line group with g bitmap lines with the same discrete dimension and adjacent continuous dimensions, that is, the target hash values ​​located in the same row in the target two-dimensional array a are grouped according to the group length g, that is, the g hash values ​​with the same discrete dimension and adjacent continuous dimensions are formed into a hash value group.

[0098] Assume that the horizontal shift line group size is 40, see Figure 5As shown, each row of horizontally shifted bitmap lines can be divided into 48 groups, each group of bitmap line groups contains 40 horizontally shifted bitmap lines, and the hash value of each group of bitmap line groups can be calculated according to hash algorithms such as MD2 and MD4 to obtain the total hash value corresponding to each group; the total hash values ​​located in the same row are stored in the corresponding row of the target total two-dimensional array ga of the current frame according to the continuous storage dimension, then the total hash values ​​located in the same row will constitute a continuous storage dimension and the total hash values ​​located in different rows will constitute a discrete storage dimension; for example, the total hash values ​​of the first row are stored in the first row of the target total two-dimensional array ga in sequence, and the total hash values ​​of the second row are stored in the second row of the target total two-dimensional array ga in sequence, then the total hash values ​​located in the first row constitute a continuous storage dimension and the total hash values ​​of the second row also constitute a continuous storage dimension, and the total hash values ​​of each column in the first row and the total hash values ​​of each column in the second row constitute a discrete storage dimension; thus, the target total two-dimensional array ga

[17]

[48] corresponding to the current frame under horizontal displacement is obtained, that is, the target total two-dimensional array ga includes 17 rows and 48 columns.

[0099] Similarly, assuming the vertical shift line group size is 20, see Figure 6 As shown, each column of the vertical shift bitmap can be divided into 54 groups, and the total target two-dimensional array ga

[15]

[54] corresponding to the current frame under the vertical shift, that is, the total target two-dimensional array ga includes 15 rows and 54 columns.

[0100] Then, based on the target total two-dimensional array ga, a target total hash table array gv with continuous storage dimensions corresponding to the current frame is constructed. Since the construction method and principle of the total hash table arrays corresponding to horizontal displacement and vertical displacement are similar, for the sake of simplicity of description, the construction of the total hash table array will be explained below using horizontal displacement as an example: the row number in the target total two-dimensional array ga is used as the element index value of the target total hash table array gv; based on the hash values ​​of the g bitmap lines in the bitmap line group with discrete dimension y and continuous dimension gx, a hash calculation is performed to obtain a total hash value, and the total hash value is used as the total hash value gh of the bitmap line group; then the total hash value gh is stored in the element ga[y][gx] at the yth row and gxth column in the target total two-dimensional array ga, and (gh, gx) is stored in the hash table gv[y] of the yth element in the target total hash table array gv as a key-value pair. It can be understood that the hash table gv[y] of the yth element in the target total hash table array gv stores 48 total hash values ​​corresponding to the yth row in the target total two-dimensional array ga.

[0101] Furthermore, in one embodiment, the method further includes:

[0102] For each target total hash value in the target total two-dimensional array, when it is detected that there is a bitmap line whose processing method is encoding in the bitmap line corresponding to the target total hash value, it is determined whether there is a target frame in the previous N frames, and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner, and the first total hash value has the same row number as the target total hash value and the column number of the first total hash value is less than the column number of the target total hash value, wherein N is a positive integer;

[0103] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0104] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as copy cache and the displacement value is the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0105] For example, in this embodiment, after the comparison between the current frame and the previous frame is completed, if there are still unsuccessful matching bitmap lines (that is, the bitmap line processing method is encoding), the query matching will continue to be performed with the bitmap line group as a unit. Figure 7 As shown, the bitmap line groups of the current frame are traversed in the order of continuous dimensions first and then discrete dimensions, and it is determined whether there are bitmap lines whose processing method is encoding in each bitmap line group, that is, whether there are bitmap lines whose processing method is encoding in the bitmap lines corresponding to each target total hash value in the target total two-dimensional array ga. If there are bitmap lines whose processing method is encoding, cache matching is required.

[0106] Assuming that the target bitmap line group to be cached is the value of the element ga[y][gx] with a discrete dimension of y and a continuous dimension of gx (i.e., the target total hash value) corresponding to the target bitmap line group, then it is determined in sequence whether there is a frame in the previous N frames corresponding to the value of the element in the total two-dimensional array with a discrete dimension of y and a continuous dimension of gx-1. All bitmap lines in the bitmap line group are processed in a copy cache. For example, it is determined that the value of the element fga[y][gx-1] with a discrete dimension of y and a continuous dimension of gx-1 in the first total two-dimensional array fga corresponding to the target frame with a cache frame index of f (i.e., the first total hash value) is Check whether the processing mode of all bitmap lines in the corresponding bitmap line group is copy cache; if all are copy cache and the displacement value of the target frame is go, determine whether the value of the element fga[y][gx+go] with discrete dimension y and continuous dimension gx+go in the first total two-dimensional array fga (that is, the second total hash value) is equal to the value of ga[y][gx]; if they are equal, record that the processing mode of all bitmap lines in the target bitmap line group is copy cache, the cache frame index is f and the displacement value is go, so that the client can copy the data corresponding to the second total hash value according to the processing mode and the displacement value.

[0107] Furthermore, in one embodiment, if there is no target frame in the first N frames and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed by copying the cache or the second total hash value is different from the target total hash value, the method further includes:

[0108] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0109] If so, determine the second column number according to the key-value pair corresponding to the third total hash value, and determine that the processing method of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column number and the column number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0110] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0111] For example, see Figure 7As shown, if there is no target frame in the first N frames, the processing method of all bitmap lines corresponding to the first total hash value in the first total two-dimensional array is copy cache, or the value of fga[y][gx+go] is not equal to the value of ga[y][gx], the yth hash table gv[y] in all the first total hash table arrays corresponding to the first N-1 frames will be searched in turn to see if there is a key equal to the value of ga[y][gx]; if there is an equal key (i.e., the third total hash value) in the yth hash table f1gv[y] in the first total hash table array corresponding to the cache frame index f1 and the value corresponding to the key is vgx (i.e., the second column number), then the processing method of all bitmap lines in the target bitmap line group is recorded as copy cache, the cache frame index is f1 and the displacement value is vgx-gx, so that the client can copy the data corresponding to the third total hash value according to the processing method and the displacement value.

[0112] It can be seen that, in this embodiment, when comparing the current frame with the previous N-1 frames, a more cost-effective coarse-grained unit bitmap line group is used as the minimum unit of cache query, and the bitmap line group of the current frame will only be compared with the bitmap line group of the previous N-1 frames with the same discrete storage dimension. The number of comparisons is less than that of the image block solution, which requires comparison with all blocks of the previous N frames, thereby reducing CPU consumption and image display delay.

[0113] After completing the query and matching of all the above bitmap lines, the adjacent bitmap line areas with the same processing method are merged to obtain the processing method of each rectangular area and send it to the encoding module; the encoding module encodes the bitmap area with the encoding processing method and then sends it to the protocol sending module; the protocol sending module encapsulates the current frame index, displacement direction, the processing method of each rectangular area (if the processing method is copy cache, its displacement value must be multiplied by the bitmap line group size corresponding to the displacement direction) and the encoded data in the protocol, and then sends the entire protocol data to the client, so that the client can obtain the image of the current frame after performing the specified processing on each rectangular area according to the protocol content, thereby obtaining the image of the remote desktop.

[0114] In this embodiment, the client includes a protocol parsing module, a displacement copy module, a cache copy module, a decoding module, and a display module. If the protocol parsing module receives stream mode coded data, it will directly send it to the decoding module for decoding and copying to the current canvas for display; if it is assumed that the protocol data with a frame index of m and a displacement direction of vertical displacement is received, the protocol parsing will be performed to obtain the processing method of each rectangular area and the encoded data; the displacement copy module copies the moving rectangular area, see Figure 8As shown, assuming that the displacement value is o, the upper left vertex of the moved rectangular area is (x, y) and the size of the rectangle is w×h, then the rectangular area with the upper left vertex (x, y+o) and size w×h on the canvas with the cache frame index m-1 is copied to the rectangular area with the upper left vertex (x, y) and size w×h on the current canvas.

[0115] The cache copy module copies the rectangular area of ​​the cache. Figure 8 As shown, assuming that the displacement value is o1, the cache frame index is f, the upper left vertex of the rectangular area is (x1, y1) and the size of the rectangle is w1×h1, the rectangular area with the upper left vertex (x1, y1+o1) and the size of w1×h1 on the canvas with the cache frame index f is copied to the rectangular area with the upper left vertex (x1, y1) and the size of w1×h1 on the current canvas; the decoding module decodes the encoded rectangular area, and copies the decoded bitmap to the rectangular area with the upper left vertex (x2, y2) and the size of w2×h2 on the current canvas; the display module copies the current canvas to the canvas with the frame index m, and hands the current canvas to the operating system for display on the monitor.

[0116] In summary, when the current frame is compared with the previous frame, the present embodiment uses a bitmap line as the minimum unit of cache query. Compared with the traditional image block solution, the comparison area is smaller, so there is a higher hit rate (especially in the word document scenario), and the occupied network bandwidth will be smaller. In addition, the bitmap line of the current frame will only be compared with the bitmap line of the previous frame with the same discrete storage dimension, and the number of comparisons is less than the image block solution that needs to be compared with all the blocks of the previous N frames, so that the CPU consumption is smaller and the image display delay is lower, and the bandwidth of most high hit rate desktop scenarios can be reduced with higher cost performance. In addition, when the current frame is compared with the previous n-1 frames, the present embodiment uses a more cost-effective coarse-grained unit bitmap line group as the minimum unit of cache query, and the bitmap line group of the current frame will only be compared with the bitmap line group of the previous N-1 frames with the same discrete storage dimension, and the number of comparisons is less than the image block solution that needs to be compared with all the blocks of the previous N frames, which will also result in lower CPU consumption and lower image display delay.

[0117] In a second aspect, an embodiment of the present application also provides a remote desktop frame query device.

[0118] In one embodiment, the remote desktop frame query device includes:

[0119] A displacement identification module, which is used to divide the current frame into a plurality of target bitmap lines corresponding to the displacement direction thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0120] A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0121] Furthermore, when the displacement direction is horizontal displacement, hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension; when the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

[0122] Furthermore, when the displacement direction is horizontal displacement, the displacement identification module is further used to:

[0123] The target hash value in the target two-dimensional array and the column number of the column where it is located are stored in the target hash table corresponding to the row where the target hash value is located in the target hash table array corresponding to the current frame in the form of key-value pairs.

[0124] Furthermore, the displacement matching module is also used for:

[0125] If the first hash value is different from the target hash value, determining whether a processing mode of a bitmap line corresponding to a second hash value in the target two-dimensional array is movement, wherein the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value is located before the target hash value;

[0126] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, the third hash value and the target hash value having the same row number and a column number of the third hash value being determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0127] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0128] Furthermore, if the processing mode of the bitmap line corresponding to the second hash value is not shifting or the third hash value is different from the target hash value, the displacement matching module is further used to:

[0129] Determine whether there is a fourth hash value that is the same as the target hash value in the target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0130] If so, determine the first column number according to the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is movement, and the displacement value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0131] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0132] Furthermore, the remote desktop frame query device also includes a cache matching module, which is used to:

[0133] The target hash values ​​in the same row of the target two-dimensional array are grouped according to a preset grouping length to obtain a plurality of hash value groups and calculate a total hash value of each hash value group;

[0134] All total hash values ​​are stored in a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0135] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row where the target total hash value is located in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0136] Furthermore, the cache matching module is also used to:

[0137] For each target total hash value in the target total two-dimensional array, when it is detected that there is a bitmap line whose processing method is encoding in the bitmap line corresponding to the target total hash value, it is determined whether there is a target frame in the previous N frames, and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner, and the first total hash value has the same row number as the target total hash value and the column number of the first total hash value is less than the column number of the target total hash value, wherein N is a positive integer;

[0138] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0139] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as copy cache and the displacement value is the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0140] Further, if there is no target frame in the first N frames and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed by copying the cache or the second total hash value is different from the target total hash value, the cache matching module is further used to:

[0141] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0142] If so, determine the second column number according to the key-value pair corresponding to the third total hash value, and determine that the processing method of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column number and the column number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0143] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0144] Furthermore, the remote desktop frame query device also includes a scene recognition module, which is used to:

[0145] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0146] If yes, the displacement identification module is made to execute the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0147] If not, the encoding module is enabled to encode the current frame in stream mode.

[0148] Among them, the functional implementation of each module in the above-mentioned remote desktop frame query device corresponds to the various steps in the above-mentioned remote desktop frame query method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0149] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0150] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0151] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0152] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0154] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A remote desktop frame query method, characterized in that: The remote desktop frame query method comprises: Divide the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line; Store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; For each target hash value in the target two-dimensional array, query and determine whether a first hash value in a first two-dimensional array corresponding to a previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; If they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

2. The remote desktop frame query method according to claim 1, characterized in that: When the displacement direction is horizontal, the hash values ​​in the same row constitute a continuous storage dimension and the hash values ​​in different rows constitute a discrete storage dimension; When the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

3. The remote desktop frame query method according to claim 2, characterized in that: When the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, the method further includes: The target hash value in the target two-dimensional array and the column number of the column where it is located are stored in the target hash table corresponding to the row where the target hash value is located in the target hash table array corresponding to the current frame in the form of key-value pairs.

4. The remote desktop frame query method according to claim 3, characterized in that: After the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the method further includes: If the first hash value is different from the target hash value, determining whether a processing mode of a bitmap line corresponding to a second hash value in the target two-dimensional array is movement, wherein the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value is located before the target hash value; If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, the third hash value and the target hash value having the same row number and a column number of the third hash value being determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value; If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

5. The remote desktop frame query method according to claim 4, characterized in that: If the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes: Determine whether there is a fourth hash value that is the same as the target hash value in the target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value; If so, determine the first column number according to the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is movement, and the displacement value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value; If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

6. The remote desktop frame query method according to claim 5, characterized in that: The method further comprises: The target hash values ​​in the same row of the target two-dimensional array are grouped according to a preset grouping length to obtain a plurality of hash value groups and calculate a total hash value of each hash value group; All total hash values ​​are stored in a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension; The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row where the target total hash value is located in the target total hash table array corresponding to the current frame in the form of key-value pairs.

7. The remote desktop frame query method according to claim 6, characterized in that: The method further comprises: For each target total hash value in the target total two-dimensional array, when it is detected that there is a bitmap line whose processing method is encoding in the bitmap line corresponding to the target total hash value, it is determined whether there is a target frame in the previous N frames, and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner, and the first total hash value has the same row number as the target total hash value and the column number of the first total hash value is less than the column number of the target total hash value, wherein N is a positive integer; If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame; If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as copy cache and the displacement value is the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

8. The remote desktop frame query method according to claim 7, characterized in that: If there is no target frame in the first N frames, all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache or the second total hash value is different from the target total hash value, the method further includes: Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value; If so, determine the second column number according to the key-value pair corresponding to the third total hash value, and determine that the processing method of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column number and the column number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value; If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

9. The remote desktop frame query method according to claim 1, characterized in that: Before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line, the method further includes: Determine whether the image scene corresponding to the current frame is a preset high hit rate scene; If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line; If not, the current frame is encoded in streaming mode.

10. A remote desktop frame query device, characterized in that: The remote desktop frame query device comprises: A displacement identification module, which is used to divide the current frame into a plurality of target bitmap lines corresponding to the displacement direction thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

Images