Remote desktop frame query method and device

Abstract

A remote desktop frame query method and device relates to the field of remote desktops. The method comprises dividing a current frame into a plurality of target bitmap lines corresponding to displacement directions according to the pixel lengths of the bitmap lines and calculating hash values ​​of the target bitmap lines; storing the hash values ​​in a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; for each target hash value in the target two-dimensional array, querying and determining whether a first hash value in a first two-dimensional array of a previous frame having the same row and column sequence number as the target hash value is the same as the target hash value; and if the first hash value is the same as the target hash value, determining that the processing mode of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value, thereby effectively reducing bandwidth usage, CPU consumption and image display delay.

Description

Technical Field

[0001] The present application relates to the field of remote desktop technology, and in particular to a remote desktop frame query method and device. Background Art

[0002] In the remote desktop world, when remote desktop software transmits the display image of desktop A to desktop B over the network for display, to reduce bandwidth usage, each frame of the desktop image is typically divided into multiple blocks. A hash value is calculated for each block, and the hash value is stored in a cached hash table for quick lookup. Therefore, with each new frame of the desktop image, the cached hash table is searched for blocks with the same hash value. If no block with the same hash value is found, the block is encoded and sent to the client. If a block with the same hash value is found, the block does not need to be encoded, and the client is directly notified to copy the block's data from the locally cached blocks to a certain area for display.

[0003] However, since each block must be compared with all blocks in the previous n frames, there are problems such as high CPU (Central Processing Unit) usage and high desktop display delay, which in turn affects user experience. Summary of the Invention

[0004] The present application provides a remote desktop frame query method and device, which can solve the technical problems of high CPU occupancy and high desktop display delay in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a remote desktop frame query method, the remote desktop frame query method comprising:

[0006] Divide the current frame into a plurality of target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line;

[0007] Store all hash values ​​into a target two-dimensional array, where hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or, hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0008] For each target hash value in the target two-dimensional array, query and determine whether a first hash value in a first two-dimensional array corresponding to a previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value;

[0009] If they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0010] In combination with the first aspect, in one embodiment, when the displacement direction is horizontal displacement, the hash values ​​located in the same row constitute a continuous storage dimension and the hash values ​​located in different rows constitute a discrete storage dimension; when the displacement direction is vertical displacement, the hash values ​​located in the same column constitute a continuous storage dimension and the hash values ​​located in different columns constitute a discrete storage dimension.

[0011] In combination with the first aspect, in one embodiment, when the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, it also includes: storing the target hash value in the target two-dimensional array and the column number of the column in which it is located in the target hash table corresponding to the row where the target hash value is located in the target hash table array corresponding to the current frame in the form of a key-value pair.

[0012] In conjunction with the first aspect, in one embodiment, after the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the method further includes:

[0013] If the first hash value is different from the target hash value, determining whether a bitmap line corresponding to a second hash value in the target two-dimensional array is processed as a move, where the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value precedes the target hash value;

[0014] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, wherein the third hash value and the target hash value have the same row number and the column number of the third hash value is determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0015] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0016] In combination with the first aspect, in one embodiment, if the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes:

[0017] Determine whether there is a fourth hash value that is the same as the target hash value in a target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0018] If so, determine the first column number based on the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is shift, and the shift value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0019] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0020] In combination with the first aspect, in one embodiment, the method further includes:

[0021] Grouping target hash values ​​in the same row of the target two-dimensional array according to a preset grouping length to obtain multiple hash value groups and calculating a total hash value of each hash value group;

[0022] Store all total hash values ​​into a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0023] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row of the target total hash value in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0024] In combination with the first aspect, in one embodiment, the method further includes:

[0025] For each target total hash value in the target total two-dimensional array, when it is detected that a bitmap line corresponding to the target total hash value has a processing mode of encoding, determining whether there is a target frame in the previous N frames in which all bitmap lines corresponding to a first total hash value in a first total two-dimensional array have a processing mode of copy caching, the first total hash value and the target total hash value have the same row number and the column number of the first total hash value is less than the column number of the target total hash value, where N is a positive integer;

[0026] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0027] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as a copy cache with a displacement value of the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0028] In conjunction with the first aspect, in one embodiment, if there is no target frame in the previous N frames and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner or the second total hash value is different from the target total hash value, the method further includes:

[0029] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0030] If so, determine the second column sequence number based on the key-value pair corresponding to the third total hash value, and determine that the processing mode of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column sequence number and the column sequence number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0031] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0032] In combination with the first aspect, in one embodiment, before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel lengths and calculating the hash value of each target bitmap line, the method further includes:

[0033] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0034] If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0035] If not, the current frame is encoded according to the streaming mode.

[0036] In a second aspect, an embodiment of the present application provides a remote desktop frame query device, the remote desktop frame query device comprising:

[0037] a displacement identification module, configured to divide the current frame into a plurality of target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and calculate a hash value for each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or wherein hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0038] A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is to move, so that the client can use the data corresponding to the first hash value as the data of the target hash value.

[0039] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0040] The current frame is divided into multiple target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and a hash value of each target bitmap line is calculated; all hash values ​​are stored in a target two-dimensional array, wherein hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension, or hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension; for each target hash value in the target two-dimensional array, query and determine whether the first hash value with the same row and column sequence as the target hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value; if the same, determine that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client can use the data corresponding to the first hash value as the data of the target hash value to realize data query of the current frame. When comparing the current frame with the previous frame, this application uses bitmap lines as the smallest unit of cache query. Compared with the traditional image block solution, the comparison area is smaller and has a higher hit rate, so the occupied network bandwidth will be smaller; and because the bitmap lines of the current frame will only be compared with the bitmap lines with the same discrete storage dimensions of the previous frame, the number of comparisons is less than the image block solution that requires comparison with all blocks of the previous n frames, which reduces CPU consumption and reduces image display delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flowchart of an embodiment of the remote desktop frame query method of the present application;

[0042] Figure 2 This is a schematic diagram of the horizontal bit shift line involved in the embodiment of the present application;

[0043] Figure 3 A schematic diagram of a vertical bit shift line involved in an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the displacement matching process involved in the embodiment of this application;

[0045] Figure 5 This is a schematic diagram of a horizontal bit shift line group involved in the embodiment of the present application;

[0046] Figure 6 Schematic diagram of a vertical shifted bitmap line group involved in the embodiment of the present application;

[0047] Figure 7 This is a schematic diagram of the cache matching process involved in the embodiment of the present application;

[0048] Figure 8 This is a schematic diagram of the canvas involved in the embodiment of this application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0051] In a first aspect, an embodiment of the present application provides a remote desktop frame query method.

[0052] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the remote desktop frame query method of this application. Figure 1 As shown, the remote desktop frame query method includes:

[0053] Step S10: Divide the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line.

[0054] Exemplarily, in this embodiment, a bitmap line refers to a line composed of multiple pixels in a bitmap, wherein the pixel length of the bitmap line can be determined according to actual needs and is not limited here. For example, if the pixel length of the bitmap line is preset to M, the number of pixels corresponding to each bitmap line is M, that is, M consecutive pixels constitute a bitmap line. In this embodiment, the displacement recognition module in the server will use a fast algorithm to compare the bitmap information of the current frame and the previous frame to estimate whether the current frame is horizontally displaced or vertically displaced relative to the previous frame, and divide the current frame into multiple horizontally displaced bitmap lines or vertically displaced bitmap lines based on the recognition result. If there is no displacement, the displacement direction of the current frame is determined according to the displacement direction of the previous frame. For example, if the displacement direction of the previous frame is horizontal displacement, the displacement direction of the current frame is also horizontal displacement. If the displacement direction of the previous frame is vertical displacement, the displacement direction of the current frame is also vertical displacement.

[0055] For horizontal displacement, each column of pixels in the current frame will be divided according to the preset bitmap line pixel length M, so that each column is divided into multiple target bitmap lines, and the number of target bitmap lines in each row is the same as the number of pixels in a row; for example, if the total number of pixels in each row of the current frame image is S and the total number of pixels in each column is P, then the number of target bitmap lines corresponding to a column is The number of target bitmap lines corresponding to one row is S, so the total number of target bitmap lines in the horizontal displacement direction of the current frame is

[0056] For vertical displacement, each row of pixels in the current frame will be divided according to the preset bitmap line pixel length M, so that each row is divided into multiple target bitmap lines, and the number of target bitmap lines in each column is the same as the number of pixels in a column; for example, if the total number of pixels in each row of the current frame image is S and the total number of pixels in each column is P, then the number of target bitmap lines corresponding to one row is The number of target bitmap lines corresponding to a column is P. Therefore, the total number of target bitmap lines in the vertical displacement direction of the current frame is

[0057] After the bitmap line division is completed, the corresponding hash value will be calculated according to the RGBA (RGBA represents the color space of red, green, blue and alpha) value of each target bitmap line.

[0058] Furthermore, in one embodiment, before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel lengths and calculating the hash value of each target bitmap line, the method further includes:

[0059] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0060] If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0061] If not, the current frame is encoded according to the streaming mode.

[0062] For example, it is understandable that most high-hit-rate desktop scenarios only feature horizontal or vertical displacement (such as browsing web pages and documents), while a small number of high-hit-rate desktop scenarios feature both horizontal and vertical displacement (such as dragging windows and images). For a small number of high-hit-rate desktop scenarios and low-hit-rate desktop scenarios (such as playing videos), using cache query to reduce network bandwidth is an extremely cost-effective method. Therefore, in this embodiment, different frame query methods are used for different cache hit rate scenarios.

[0063] Specifically, the server in this embodiment also includes a bitmap capture module, a scene recognition module, a displacement matching module, a cache matching module, an encoding module, and a protocol sending module. The bitmap capture module captures the bitmap information of the desktop image each time the user's desktop image changes, and sends it to the scene recognition module. The scene recognition module performs scene recognition based on the bitmap information. If the current scene is a desktop scene with a high hit rate, the bitmap information of the desktop image is sent to the displacement recognition module for cache query. If the current scene is a desktop scene with a small number of high hit rates or a desktop scene with a low hit rate, the bitmap information of the desktop image is sent to the encoding module, so that the encoding module directly performs encoding processing in streaming mode and sends it to the client through the protocol sending module.

[0064] It should be noted that the desktop scene type of the current frame can be determined based on the desktop scene type of the previous frame. For example, if the previous frame is a preset desktop scene with a majority of high hit rates, then the current frame is also a preset desktop scene with a majority of high hit rates. If the previous frame is a desktop scene with a low hit rate, then the current frame is also a desktop scene with a low hit rate. In addition, during the frame query process for the current frame, the desktop scene type of the current frame is also updated and calculated to provide a basis for the desktop scene type of the next frame. It should be understood that the calculation of desktop scenes with a majority of high hit rates, desktop scenes with a minority of high hit rates, and desktop scenes with a low hit rate is common knowledge in the art and will not be elaborated on here.

[0065] It can be seen that this embodiment mainly optimizes the bandwidth occupancy and CPU occupancy of the above-mentioned most high-hit-rate desktop scenarios. When a small number of high-hit-rate desktop scenarios and low-hit-rate desktop scenarios are identified, it can automatically switch to streaming mode to encode the desktop image in full screen, thereby reducing CPU occupancy and delay; and when most high-hit-rate desktop scenarios are identified, it can automatically switch to cache mode to reduce bandwidth occupancy and CPU occupancy.

[0066] Step S20: All hash values ​​are stored in a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension.

[0067] Specifically, when the displacement direction is horizontal displacement, hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension;

[0068] When the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

[0069] Wherein, when the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, the method further includes:

[0070] The target hash value and the column number of the column in the target two-dimensional array are stored in the target hash table corresponding to the row of the target hash value in the target hash table array corresponding to the current frame in the form of key-value pairs.

[0071] For example, in this embodiment, the displacement recognition module constructs a two-dimensional array and a hash table array for each frame to store hash values ​​and hash tables, respectively. It should be noted that since the data storage method and principle are the same for each frame, for simplicity of description, the following embodiment will use the data storage of the current frame as an example to illustrate the assignment of values ​​to the two-dimensional array and hash table array.

[0072] Specifically, when assigning values ​​to a two-dimensional array, if the displacement direction of the current frame image is horizontal displacement, the hash values ​​located in the same row will be stored according to a continuous storage dimension, and the hash values ​​located in the same row of the current frame image will constitute a continuous storage dimension, while the hash values ​​located in different rows will constitute a discrete storage dimension; if the displacement direction of the current frame image is vertical displacement, the hash values ​​located in the same column of the current frame image will be stored according to a continuous storage dimension, and the hash values ​​located in the same column will constitute a continuous storage dimension, while the hash values ​​located in different columns will constitute a discrete storage dimension.

[0073] Assuming the desktop resolution is 1920×1080, the horizontal displacement corresponds to a bitmap line length of 64 pixels, and the vertical displacement corresponds to a bitmap line length of 128 pixels; see Figure 2 As shown, when the displacement recognition result of the current frame is horizontal displacement, the displacement recognition module will divide each column according to 64 pixel units, and the desktop image will be divided into 17 rows and 1920 columns, among which the target bitmap line length of the first 16 rows is 64, and the length of the target bitmap line of the 17th row is 56; then the hash values ​​of the 1920 target bitmap lines (i.e., horizontal displacement bitmap lines) corresponding to each row are stored in the corresponding rows of the target two-dimensional array a according to the continuous storage dimension. For example, the hash value of the first row in the current frame image is stored in the first row of the target two-dimensional array a in sequence, and the hash value of the second row in the current frame image is stored in the first row of the target two-dimensional array a in sequence. The hash values ​​of different rows in the target two-dimensional array a are sequentially stored in the second row, and so on. Then, the hash values ​​of different rows in the target two-dimensional array a will constitute discrete storage dimensions, forming the target two-dimensional array a

[17]

[1920] , that is, the target two-dimensional array a includes 17 rows and 1920 columns. It can be understood that the target bitmap line hash value corresponding to the y-th row and x-th column in the current frame is stored in the element a[y][x] of the y-th row and x-th column in the target two-dimensional array a

[17]

[1920] , and for the target two-dimensional array a

[17]

[1920] , the element of the y-th row and x-th column can also refer to the element with discrete dimension y and continuous dimension x.

[0074] Similarly, see Figure 3 As shown, when the displacement recognition result of the current frame is vertical displacement, the displacement recognition module will divide each row according to 128 pixels, and the desktop image will be divided into 1080 rows and 15 columns, where the target bitmap line length of each column is 128; then the hash values ​​of the 1080 target bitmap lines (i.e., vertical displacement bitmap lines) corresponding to each column are stored in the corresponding row of the target two-dimensional array a according to the continuous storage dimension. For example, the hash value of the first column in the current frame image is stored in the first row of the target two-dimensional array a in sequence. , the hash values ​​of the second column in the current frame image are sequentially stored in the second row of the target two-dimensional array a, and so on. The hash values ​​on different columns in the target two-dimensional array a will constitute discrete storage dimensions, forming the target two-dimensional array a

[15]

[1080] , that is, the target two-dimensional array a includes 15 rows and 1080 columns; it can be understood that the target bitmap line hash value corresponding to the y-th row and x-th column in the current frame is stored in the element a[x][y] of the x-th row and y-th column in the target two-dimensional array a

[15]

[1080] .

[0075] After completing the assignment of the two-dimensional array, the hash table array will also be assigned. Since the hash table array assignment method and principle corresponding to horizontal and vertical shifts are similar, for the sake of simplicity, the following will use horizontal shift as an example to illustrate the hash table array assignment: the row number in the target two-dimensional array a is used as the element index value of the target hash table array v, so that the number of elements in the target hash table array v is consistent with the number of rows in the target two-dimensional array a; then, according to the continuous storage dimension, each hash value of each row in the target two-dimensional array a and the column number of the column it belongs to are stored in the hash table at the corresponding element index value in the target hash table array v as a key-value pair; where, in each key-value pair, the hash value is the key and the column number is the value, and each element in the target hash table array v is a hash table. For example, if the hash value corresponding to the element a[y][x] at the yth row and xth column in the target two-dimensional array a is h, then the (h,x) key-value pair is stored in the hash table v[y] corresponding to the yth element of the target hash table array v.

[0076] It should be noted that if the displacement direction of the current frame is different from that of the previous frame, the two-dimensional array and hash table array of the previous frame need to be reconstructed according to the displacement direction of the current frame.

[0077] Step S30: for each target hash value in the target two-dimensional array, query and determine whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value.

[0078] Exemplarily, in this embodiment, a bitmap line processing method table is first initialized by the displacement matching module in the server to record the processing method to be performed on each target bitmap line of the current frame, with the default value being no processing; specifically, the target two-dimensional array of the current frame is matched with the previous frame in the order of first continuous storage dimension and then discrete storage dimension, and then the processing method of each bitmap line is set according to the matching result.

[0079] Since the frame query method and principle of each target hash value in the target two-dimensional array are similar regardless of horizontal displacement or vertical displacement, for the sake of simplicity of description, the following embodiment will take one of the target hash values ​​under horizontal displacement as an example to illustrate the frame query of the corresponding target bitmap line. It is assumed that a frame query is required for the target bitmap line corresponding to the target hash value of the element a[y][x] with discrete dimension y and continuous dimension x in the target two-dimensional array a corresponding to the current frame: determine whether the value of the element la[y][x] with discrete dimension y and continuous dimension x in the first two-dimensional array la corresponding to the previous frame (i.e., the first hash value) is equal to the value of the element a[y][x] in the target two-dimensional array a (i.e., the target hash value), and then determine whether the frame query of the target bitmap line corresponding to the target hash value is completed based on the judgment result.

[0080] It can be seen that, in this embodiment, when comparing the current frame with the previous frame, the bitmap line is used as the smallest unit of cache query. Compared with the traditional image block solution, the comparison area is smaller and has a higher hit rate, so that the occupied network bandwidth will be smaller; and because the bitmap line of the current frame will only be compared with the bitmap line of the previous frame with the same discrete storage dimension, the number of comparisons is less than the image block solution that requires comparison with all blocks of the previous n frames, so that the CPU consumption is smaller and the image display delay is lower.

[0081] Step S40: If they are the same, determining that the processing method of the target bitmap line corresponding to the target hash value is moving, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

[0082] Exemplarily, in this embodiment, if the first hash value (i.e., the value corresponding to la[y][x]) is equal to the target hash value (i.e., the value corresponding to a[y][x]), it means that the frame query of the target bitmap line corresponding to the target hash value has been completed, and the processing method of the target bitmap line corresponding to the target hash value is recorded as moving, so that the client can copy the data corresponding to the first hash value according to the processing method, thereby completing the frame query of one bitmap line and continuing with the frame query processing of the next bitmap line.

[0083] Furthermore, in one embodiment, after the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the method further includes:

[0084] If the first hash value is different from the target hash value, determining whether a bitmap line corresponding to a second hash value in the target two-dimensional array is processed as a move, where the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value precedes the target hash value;

[0085] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, wherein the third hash value and the target hash value have the same row number and the column number of the third hash value is determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0086] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0087] For example, see Figure 4As shown, if the first hash value (i.e., the value corresponding to la[y][x]) is not equal to the target hash value (i.e., the value corresponding to a[y][x]), it will continue to determine whether the bitmap line processing method corresponding to the value of the element a[y][x-1] with discrete dimension y and continuous dimension x-1 in the target two-dimensional array a corresponding to the current frame (i.e., the second hash value) is moving. If it is moving and the displacement value is o, it is determined whether the value of the element la[y][x+o] with discrete dimension y and continuous dimension x+o in the first two-dimensional array la corresponding to the previous frame (i.e., the third hash value) is equal to the value of a[y][x]. If they are equal, it is recorded that the processing method of the target bitmap line with discrete dimension y and continuous dimension x in the current frame is moving and the displacement value is o, so as to complete the frame query of the target bitmap line, so that the client can copy the data corresponding to the third hash value according to the processing method and the displacement value.

[0088] Furthermore, in one embodiment, if the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes:

[0089] Determine whether there is a fourth hash value that is the same as the target hash value in a target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0090] If so, determine the first column number based on the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is shift, and the shift value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0091] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0092] For example, see Figure 4As shown, if the bitmap line processing method corresponding to the value of a[y][x-1] is not move, or the value of la[y][x+o] is not equal to the value of a[y][x], the query will continue to check whether there is a key (i.e., the fourth hash value) equal to the value of a[y][x] in the y-th hash table lv[y] in the first hash table array lv corresponding to the previous frame. If there are equal keys and the continuous storage dimension index of the corresponding hash value is lx (i.e., the first column sequence number), then the processing method of the target bitmap line with discrete dimension y and continuous dimension x in the current frame is recorded as move and the displacement value is lx-x to complete the frame query of the target bitmap line, so that the client can copy the data corresponding to the fourth hash value according to the processing method; if there are no equal keys, then the processing method of the target bitmap line with discrete dimension y and continuous dimension x in the current frame is recorded as encoding.

[0093] Furthermore, in one embodiment, the method further includes:

[0094] Grouping target hash values ​​in the same row of the target two-dimensional array according to a preset grouping length to obtain multiple hash value groups and calculating a total hash value of each hash value group;

[0095] Store all total hash values ​​into a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0096] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row of the target total hash value in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0097] For example, it should be noted that the group length refers to the number of bitmap lines (i.e., hash values) contained in a bitmap line group (i.e., hash value group). For example, if the group length is g, then a bitmap line group contains g bitmap lines, that is, a hash value group contains g hash values. The specific value setting can be determined according to actual needs and is not limited here. In this embodiment, the cache matching module groups the bitmap lines according to the group length g in the continuous dimension to form a bitmap line group with g bitmap lines with the same discrete dimension and adjacent continuous dimensions, that is, the target hash values ​​located in the same row in the target two-dimensional array a are grouped according to the group length g, that is, the g hash values ​​with the same discrete dimension and adjacent continuous dimensions are formed into a hash value group.

[0098] Assume that the horizontal shift line group size is 40, see Figure 5As shown, each row of horizontally shifted bitmap lines can be divided into 48 groups, each group of bitmap line groups contains 40 horizontally shifted bitmap lines, and the hash value of each group of bitmap line groups can be calculated according to hash algorithms such as MD2 and MD4 to obtain the total hash value corresponding to each group; the total hash values ​​located in the same row are stored in the corresponding row of the target total two-dimensional array ga of the current frame according to the continuous storage dimension, then the total hash values ​​located in the same row will constitute a continuous storage dimension and the total hash values ​​located in different rows will constitute a discrete storage dimension; for example, the total hash values ​​of the first row are stored in the first row of the target total two-dimensional array ga in sequence, and the total hash values ​​of the second row are stored in the second row of the target total two-dimensional array ga in sequence, then the total hash values ​​located in the first row constitute a continuous storage dimension and the total hash values ​​of the second row also constitute a continuous storage dimension, and the total hash values ​​of each column in the first row and the total hash values ​​of each column in the second row constitute a discrete storage dimension; at this point, the target total two-dimensional array ga corresponding to the current frame under horizontal shift is obtained

[17]

[48] , that is, the target total two-dimensional array ga includes 17 rows and 48 columns.

[0099] Similarly, assuming the vertical shift line group size is 20, see Figure 6 As shown, each column of vertical displacement bitmap lines can be divided into 54 groups, and the total target two-dimensional array ga

[15]

[54] corresponding to the current frame under vertical displacement, that is, the total target two-dimensional array ga includes 15 rows and 54 columns.

[0100] Then, based on the target total two-dimensional array ga, a target total hash table array gv with continuous storage dimensions corresponding to the current frame is constructed. Since the construction method and principle of the total hash table arrays corresponding to horizontal displacement and vertical displacement are similar, for the sake of simplicity, the construction of the total hash table array will be explained below using horizontal displacement as an example: the row number in the target total two-dimensional array ga is used as the element index value of the target total hash table array gv; based on the hash values ​​of the g bitmap lines in the bitmap line group with discrete dimension y and continuous dimension gx, a hash calculation is performed to obtain a total hash value, and the total hash value is used as the total hash value gh of the bitmap line group; then the total hash value gh is stored in the element ga[y][gx] in the y-th row and gx-th column of the target total two-dimensional array ga, and (gh, gx) is stored in the hash table gv[y] of the y-th element in the target total hash table array gv as a key-value pair. It can be understood that the hash table gv[y] of the yth element in the target total hash table array gv stores the 48 total hash values ​​corresponding to the yth row in the target total two-dimensional array ga.

[0101] Furthermore, in one embodiment, the method further includes:

[0102] For each target total hash value in the target total two-dimensional array, when it is detected that a bitmap line corresponding to the target total hash value has a processing mode of encoding, determining whether there is a target frame in the previous N frames in which all bitmap lines corresponding to a first total hash value in a first total two-dimensional array have a processing mode of copy caching, the first total hash value and the target total hash value have the same row number and the column number of the first total hash value is less than the column number of the target total hash value, where N is a positive integer;

[0103] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0104] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as a copy cache with a displacement value of the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0105] For example, in this embodiment, after the comparison between the current frame and the previous frame is completed, if there are still unmatched bitmap lines (ie, the bitmap line processing method is encoding), the query matching will continue with the bitmap line group as a unit. Figure 7 As shown, the bitmap line groups of the current frame are traversed in the order of continuous dimensions first and then discrete dimensions, and it is determined whether there are bitmap lines with encoding processing mode in each bitmap line group, that is, whether there are bitmap lines with encoding processing mode in the bitmap lines corresponding to each target total hash value in the target total two-dimensional array ga. If there are bitmap lines with encoding processing mode, cache matching is required.

[0106] Assuming that the target bitmap line group to be cached is the value of the element ga[y][gx] with a discrete dimension of y and a continuous dimension of gx (i.e., the target total hash value) corresponding to the target bitmap line group, then it is determined in sequence whether there is a frame in the previous N frames corresponding to the total two-dimensional array with a discrete dimension of y and a continuous dimension of gx-1. All bitmap lines in the bitmap line group are processed in a copy cache. For example, the value of the element fga[y][gx-1] with a discrete dimension of y and a continuous dimension of gx-1 in the first total two-dimensional array fga corresponding to the target frame with a cache frame index of f (i.e., the first total hash value) is determined. Whether the processing mode of all bitmap lines in the corresponding bitmap line group is copy cache; if all are copy cache and the displacement value of the target frame is go, then determine whether the value of the element fga[y][gx+go] with discrete dimension y and continuous dimension gx+go in the first total two-dimensional array fga (that is, the second total hash value) is equal to the value of ga[y][gx]. If they are equal, record that the processing mode of all bitmap lines in the target bitmap line group is copy cache, the cache frame index is f and the displacement value is go, so that the client can copy the data corresponding to the second total hash value according to the processing mode and displacement value.

[0107] Furthermore, in one embodiment, if there is no target frame in the first N frames and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner or the second total hash value is different from the target total hash value, the method further includes:

[0108] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0109] If so, determine the second column sequence number based on the key-value pair corresponding to the third total hash value, and determine that the processing mode of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column sequence number and the column sequence number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0110] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0111] For example, see Figure 7As shown, if there is no target frame in the first N frames and the processing mode for all bitmap lines corresponding to the first total hash value in the first total two-dimensional array is copy cache, or the value of fga[y][gx+go] is not equal to the value of ga[y][gx], the y-th hash table gv[y] in all the first total hash table arrays corresponding to the first N-1 frames will be searched in sequence to see if there is a key equal to the value of ga[y][gx]; if there is an equal key (i.e., the third total hash value) in the y-th hash table f1gv[y] in the first total hash table array corresponding to the cache frame index f1 and the value corresponding to the key is vgx (i.e., the second column number), then the processing mode for all bitmap lines in the target bitmap line group is recorded as copy cache, the cache frame index is f1, and the displacement value is vgx-gx, so that the client can copy the data corresponding to the third total hash value according to the processing mode and the displacement value.

[0112] It can be seen that, in this embodiment, when comparing the current frame with the previous N-1 frames, a more cost-effective coarse-grained unit bitmap line group is used as the minimum unit of cache query, and the bitmap line group of the current frame is only compared with the bitmap line group of the previous N-1 frames with the same discrete storage dimension. The number of comparisons is less than that of the image block solution, which requires comparison with all blocks of the previous N frames, thereby reducing CPU consumption and image display delay.

[0113] After completing the query and matching of all the above bitmap lines, the adjacent bitmap line areas with the same processing method are merged to obtain the processing method of each rectangular area and send it to the encoding module; the encoding module encodes the bitmap area with encoding processing method and then sends it to the protocol sending module; the protocol sending module encapsulates the current frame index, displacement direction, the processing method of each rectangular area (if the processing method is copy cache, its displacement value must be multiplied by the bitmap line group size corresponding to the displacement direction) and the encoded data in the protocol, and then sends the entire protocol data to the client, so that the client can perform specified processing on each rectangular area according to the protocol content to obtain the image of the current frame, thereby obtaining the image of the remote desktop.

[0114] In this embodiment, the client includes a protocol parsing module, a displacement copy module, a cache copy module, a decoding module, and a display module. If the protocol parsing module receives stream-mode coded data, it will directly send it to the decoding module for decoding and copying to the current canvas before displaying it. If, however, the protocol data with a frame index of m and a vertical displacement direction is received, the protocol parsing will be performed to obtain the processing method and encoded data for each rectangular area. The displacement copy module copies the moving rectangular area, see Figure 8As shown, assuming that the displacement value is o, the upper left vertex of the moved rectangular area is (x, y) and the size of the rectangle is w×h, then the rectangular area with the upper left vertex (x, y+o) and the size of w×h on the canvas with the cache frame index m-1 is copied to the rectangular area with the upper left vertex (x, y) and the size of w×h on the current canvas.

[0115] The cache copy module copies the rectangular area of ​​the cache. Figure 8 As shown, assuming that the displacement value is o1, the cache frame index is f, the upper left vertex of the rectangular area is (x1, y1) and the rectangle size is w1×h1, the rectangular area with the upper left vertex (x1, y1+o1) and the size of w1×h1 on the canvas with the cache frame index f is copied to the rectangular area with the upper left vertex (x1, y1) and the size of w1×h1 on the current canvas; the decoding module decodes the encoded rectangular area and copies the decoded bitmap to the rectangular area with the upper left vertex (x2, y2) and the size of w2×h2 on the current canvas; the display module copies the current canvas to the canvas with the frame index m, and hands the current canvas to the operating system for display on the monitor.

[0116] In summary, when comparing the current frame with the previous frame, this embodiment uses a bitmap line as the minimum unit of cache query. Compared with the traditional image block solution, since the comparison area is smaller, it has a higher hit rate (especially in the word document scenario), and the occupied network bandwidth will be smaller. In addition, the bitmap line of the current frame will only be compared with the bitmap line of the previous frame with the same discrete storage dimension. The number of comparisons is less than the image block solution that requires comparison with all blocks of the previous N frames, which reduces CPU consumption and image display delay, and can more cost-effectively reduce the bandwidth of most high-hit-rate desktop scenarios. In addition, when comparing the current frame with the previous n-1 frames, this embodiment uses a more cost-effective coarse-grained unit bitmap line group as the minimum unit of cache query, and the bitmap line group of the current frame will only be compared with the bitmap line group of the previous N-1 frames with the same discrete storage dimension. The number of comparisons is less than the image block solution that requires comparison with all blocks of the previous N frames, which also reduces CPU consumption and image display delay.

[0117] In a second aspect, an embodiment of the present application also provides a remote desktop frame query device.

[0118] In one embodiment, the remote desktop frame query device includes:

[0119] a displacement identification module, configured to divide the current frame into a plurality of target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and calculate a hash value for each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or wherein hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension;

[0120] A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is to move, so that the client can use the data corresponding to the first hash value as the data of the target hash value.

[0121] Furthermore, when the displacement direction is horizontal displacement, hash values ​​located in the same row constitute a continuous storage dimension and hash values ​​located in different rows constitute a discrete storage dimension; when the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

[0122] Furthermore, when the displacement direction is horizontal displacement, the displacement identification module is further configured to:

[0123] The target hash value and the column number of the column in the target two-dimensional array are stored in the target hash table corresponding to the row of the target hash value in the target hash table array corresponding to the current frame in the form of key-value pairs.

[0124] Furthermore, the displacement matching module is also used to:

[0125] If the first hash value is different from the target hash value, determining whether a bitmap line corresponding to a second hash value in the target two-dimensional array is processed as a move, where the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value precedes the target hash value;

[0126] If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, wherein the third hash value and the target hash value have the same row number and the column number of the third hash value is determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value;

[0127] If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

[0128] Furthermore, if the processing mode of the bitmap line corresponding to the second hash value is not shift or the third hash value is different from the target hash value, the displacement matching module is further configured to:

[0129] Determine whether there is a fourth hash value that is the same as the target hash value in a target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value;

[0130] If so, determine the first column number based on the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is shift, and the shift value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value;

[0131] If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

[0132] Furthermore, the remote desktop frame query device further includes a cache matching module, which is used to:

[0133] Grouping target hash values ​​in the same row of the target two-dimensional array according to a preset grouping length to obtain multiple hash value groups and calculating a total hash value of each hash value group;

[0134] Store all total hash values ​​into a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension;

[0135] The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row of the target total hash value in the target total hash table array corresponding to the current frame in the form of key-value pairs.

[0136] Furthermore, the cache matching module is also used to:

[0137] For each target total hash value in the target total two-dimensional array, when it is detected that a bitmap line corresponding to the target total hash value has a processing mode of encoding, determining whether there is a target frame in the previous N frames in which all bitmap lines corresponding to a first total hash value in a first total two-dimensional array have a processing mode of copy caching, the first total hash value and the target total hash value have the same row number and the column number of the first total hash value is less than the column number of the target total hash value, where N is a positive integer;

[0138] If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame;

[0139] If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as a copy cache with a displacement value of the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

[0140] Furthermore, if there is no target frame in the first N frames and all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache or the second total hash value is different from the target total hash value, the cache matching module is further configured to:

[0141] Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value;

[0142] If so, determine the second column sequence number based on the key-value pair corresponding to the third total hash value, and determine that the processing mode of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column sequence number and the column sequence number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value;

[0143] If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

[0144] Furthermore, the remote desktop frame query device further includes a scene recognition module, which is used to:

[0145] Determine whether the image scene corresponding to the current frame is a preset high hit rate scene;

[0146] If so, the displacement identification module is caused to execute the steps of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line;

[0147] If not, the encoding module is enabled to encode the current frame in stream mode.

[0148] Among them, the functional implementation of each module in the above-mentioned remote desktop frame query device corresponds to the various steps in the above-mentioned remote desktop frame query method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0149] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0150] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0151] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0152] In some processes described in the embodiments of this application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of this application or may be executed in parallel. The sequence numbers of the operations are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0154] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A remote desktop frame query method, characterized in that: The remote desktop frame query method includes: Divide the current frame into a plurality of target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and calculate a hash value of each target bitmap line; Store all hash values ​​into a target two-dimensional array, where hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or, hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; For each target hash value in the target two-dimensional array, query and determine whether a first hash value in a first two-dimensional array corresponding to a previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; If they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is movement, so that the client uses the data corresponding to the first hash value as the data of the target hash value.

2. The remote desktop frame query method according to claim 1, wherein: When the displacement direction is horizontal, the hash values ​​in the same row constitute a continuous storage dimension and the hash values ​​in different rows constitute a discrete storage dimension; When the displacement direction is vertical displacement, hash values ​​located in the same column constitute a continuous storage dimension and hash values ​​located in different columns constitute a discrete storage dimension.

3. The remote desktop frame query method according to claim 2, wherein: When the displacement direction is horizontal displacement, after the step of storing all hash values ​​in the target two-dimensional array, the method further includes: The target hash value and the column number of the column in the target two-dimensional array are stored in the target hash table corresponding to the row of the target hash value in the target hash table array corresponding to the current frame in the form of key-value pairs.

4. The remote desktop frame query method according to claim 3, wherein: After the step of querying and determining whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, the method further includes: If the first hash value is different from the target hash value, determining whether a bitmap line corresponding to a second hash value in the target two-dimensional array is processed as a move, where the second hash value and the target hash value are hash values ​​of two adjacent columns in the same row and the second hash value precedes the target hash value; If so, determining whether a third hash value in the first two-dimensional array is the same as the target hash value, wherein the third hash value and the target hash value have the same row number and the column number of the third hash value is determined based on the first displacement value of the bitmap line corresponding to the second hash value and the column number of the target hash value; If the third hash value is the same as the target hash value, the target bitmap line corresponding to the target hash value is determined to be moved and the displacement value is the first displacement value, so that the client uses the data corresponding to the third hash value as the data of the target hash value.

5. The remote desktop frame query method according to claim 4, wherein: If the processing mode of the bitmap line corresponding to the second hash value is not moving or the third hash value is different from the target hash value, the method further includes: Determine whether there is a fourth hash value that is the same as the target hash value in a target hash table in the first hash table array corresponding to the previous frame, and the row number corresponding to the target hash table is the same as the row number corresponding to the target hash value; If so, determine the first column number based on the key-value pair corresponding to the fourth hash value, and determine that the processing method of the target bitmap line corresponding to the target hash value is shift, and the shift value is the difference between the first column number and the column number of the column where the target hash value is located, so that the client uses the data corresponding to the fourth hash value as the data of the target hash value; If not, it is determined that the processing method of the target bitmap line corresponding to the target hash value is encoding, so that the client can encode the area data corresponding to the target hash value.

6. The remote desktop frame query method according to claim 5, wherein: The method further comprises: Grouping target hash values ​​in the same row of the target two-dimensional array according to a preset grouping length to obtain multiple hash value groups and calculating a total hash value of each hash value group; Store all total hash values ​​into a target total two-dimensional array, wherein the total hash values ​​in the same row of the target total two-dimensional array constitute a continuous storage dimension and the total hash values ​​in different rows constitute a discrete storage dimension; The target total hash value and the column number of the column in the target total two-dimensional array are stored in the target total hash table corresponding to the row of the target total hash value in the target total hash table array corresponding to the current frame in the form of key-value pairs.

7. The remote desktop frame query method according to claim 6, wherein: The method further comprises: For each target total hash value in the target total two-dimensional array, when it is detected that a bitmap line corresponding to the target total hash value has a processing mode of encoding, determining whether there is a target frame in the previous N frames in which all bitmap lines corresponding to a first total hash value in a first total two-dimensional array have a processing mode of copy caching, the first total hash value and the target total hash value have the same row number and the column number of the first total hash value is less than the column number of the target total hash value, where N is a positive integer; If so, determining whether a second total hash value in the first total two-dimensional array is the same as the target total hash value, the second total hash value and the target total hash value have the same row number and the column number of the second total hash value is determined according to the column number of the target total hash value and the second displacement value of the target frame; If the second total hash value is the same as the target total hash value, the target bitmap line group corresponding to the target total hash value is determined to be processed as a copy cache with a displacement value of the second displacement value, so that the client uses the data corresponding to the second total hash value as the data of the target total hash value.

8. The remote desktop frame query method according to claim 7, wherein: If there is no target frame in the first N frames, all bitmap lines corresponding to the first total hash value in the first total two-dimensional array are processed in a copy cache manner or the second total hash value is different from the target total hash value. The method further includes: Determine whether there is a third total hash value that is the same as the target total hash value in the target total hash table in all first total hash table arrays corresponding to the previous N-1 frames, and the row number corresponding to the target total hash table is the same as the row number corresponding to the target total hash value; If so, determine the second column sequence number based on the key-value pair corresponding to the third total hash value, and determine that the processing mode of the target bitmap line group corresponding to the target total hash value is copy cache, and the displacement value is the difference between the second column sequence number and the column sequence number of the column where the target total hash value is located, so that the client uses the data corresponding to the third total hash value as the data of the target total hash value; If not, it is determined that the processing method of the target bitmap line group corresponding to the target total hash value is encoding, so that the client can encode the area data corresponding to the target total hash value.

9. The remote desktop frame query method according to claim 1, wherein: Before the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line, the method further includes: Determine whether the image scene corresponding to the current frame is a preset high hit rate scene; If yes, then executing the step of dividing the current frame into a plurality of target bitmap lines corresponding to the displacement directions thereof according to the preset bitmap line pixel length and calculating the hash value of each target bitmap line; If not, the current frame is encoded according to the streaming mode.

10. A remote desktop frame query device, characterized in that: The remote desktop frame query device includes: a displacement identification module, configured to divide the current frame into a plurality of target bitmap lines corresponding to their displacement directions according to a preset bitmap line pixel length and calculate a hash value for each target bitmap line; store all hash values ​​into a target two-dimensional array, wherein hash values ​​in the same row constitute a continuous storage dimension and hash values ​​in different rows constitute a discrete storage dimension, or wherein hash values ​​in the same column constitute a continuous storage dimension and hash values ​​in different columns constitute a discrete storage dimension; A displacement matching module is used to query and determine, for each target hash value in the target two-dimensional array, whether the first hash value in the first two-dimensional array corresponding to the previous frame is the same as the target hash value, and the row and column numbers corresponding to the first hash value are the same as the target hash value; if they are the same, it is determined that the processing method of the target bitmap line corresponding to the target hash value is to move, so that the client can use the data corresponding to the first hash value as the data of the target hash value.

Images