Data transmission method, device, storage medium, program and system

Abstract

Provided in the embodiments of the present disclosure are a data transmission method, a device, a storage medium, a program and a system. The method comprises: partitioning a dirty region in a video picture, so as to obtain a target image block, and determining a plurality of row hash values and a block hash value of the target image block; and, if it is determined, on the basis of the block hash value, the plurality of row hash values, a block cache table and a row cache table, that the target image block is stored in a cache space, transmitting to a client target cache index information of the target image block and corresponding pixel position information of the target image block in the video picture, wherein the block cache table is used for storing block cache index information of each image block that has been cached, and the row cache table is used for storing row cache index information of different rows of pixels in each image block that has been cached. The client obtains the target image block from the cache space on the basis of the target cache index information, and displays the target image block on the basis of the pixel position information. The present disclosure improves the image block hit rate by means of two-level cache, so as to reduce bandwidth occupation. Further provided in the embodiments of the present disclosure is a method for determining the type of an image block on the basis of the color quantity and pixel quantity proportion.

Description

[0001] TECHNICAL FIELD The present disclosure relates to the field of cloud computing technology, and more particularly to a data transmission method, device, storage medium, program, and system. Background With the continuous emergence of various cloud service products, such as cloud desktops, users can utilize these cloud service products for office work, entertainment, and other activities at a lower cost and in a more flexible and convenient manner. This eliminates the need to purchase expensive host devices with greater computing power or install various required applications, especially those that consume significant computing power, on local host machines. Instead, users can simply use the corresponding services provided by cloud service providers. Cloud service providers possess abundant hardware resources and, through virtualization and other technologies, can create a large number of virtual machines (i.e., cloud computers) on which they deploy various cloud services, such as the aforementioned cloud desktops, for user use. Taking cloud desktops as an example, to use cloud desktop services, users need to install the corresponding client on their local host machine. This allows users to remotely access the cloud desktop through a transmission link between the client and the corresponding server (i.e., the transmission link between the host machine where the client resides and the virtual machine where the cloud desktop server resides). In practice, the server encodes the cloud desktop content to be displayed into a video stream and transmits it to the client, which decodes and displays the corresponding cloud desktop content. Although encoding (i.e., compressing) each frame of the cloud desktop image to be transmitted can reduce the amount of data transmitted from the server to the client and lower bandwidth requirements, based on video encoding principles, if the video image to be encoded (e.g., a cloud desktop video frame) contains a large dirty area and is frequently updated, the server still needs to encode a large amount of content, thus requiring a large bandwidth to transmit the video image (e.g., a cloud desktop video frame). A dirty area refers to the image area in the current video frame that has changed compared to the previous video frame. SUMMARY OF THE INVENTION The present disclosure provides a data transmission method, device, storage medium, program, and system for reducing the bandwidth required for video transmission.In a first aspect, embodiments of the present disclosure provide a data transmission method, applied to a server, the method comprising: performing block processing on a dirty area in a video image to be transmitted to obtain a target image block contained in the dirty area, where the target image block is any one of multiple image blocks obtained by the block processing; determining multiple row hash values ​​and block hash values ​​of the target image block, where the multiple row hash values ​​correspond to multiple rows of pixels contained in the target image block; if it is determined that the target image block is stored in a cache space based on the block hash value, the multiple row hash values, a block cache table, and a row cache table, determining target cache index information for the target image block, where the block cache table is used to store block cache index information for each cached image block, and the row cache table is used to store row cache index information for different rows of pixels in each cached image block; and sending pixel position information corresponding to the target image block in the video image and the target cache index information to a client, so that the client obtains the target image block from the cache space based on the target cache index information and displays the target image block based on the pixel position information. In a second aspect, an embodiment of the present disclosure provides a data transmission device, applied to a server, the device comprising: a blocking module configured to perform blocking processing on a dirty area in a video picture to be transmitted to obtain a target image block contained in the dirty area, where the target image block is any one of multiple image blocks obtained by the blocking processing; a determination module configured to determine multiple row hash values ​​and block hash values ​​of the target image block, where the multiple row hash values ​​correspond to multiple rows of pixels contained in the target image block; a query module configured to determine target cache index information of the target image block if it is determined that the target image block is stored in a cache space based on the block hash value, the multiple row hash values, a block cache table, and a row cache table, wherein the block cache table is used to store block cache index information of each cached image block, and the row cache table is used to store row cache index information of different rows of pixels in each cached image block; and a sending module configured to send pixel position information corresponding to the target image block in the video picture and the target cache index information to a client. The client obtains the target image block from the cache space based on the target cache index information and displays the target image block based on the pixel position information. In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the processor executes the executable code, the processor can at least implement the data transmission method described in the first aspect.In a fourth aspect, embodiments of the present disclosure provide a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is enabled to implement at least the data transmission method described in the first aspect. In a fifth aspect, embodiments of the present disclosure provide a computer program product. The computer program product includes a computer program. When executed by a processor of an electronic device, the processor is enabled to implement at least the data transmission method described in the first aspect. In a sixth aspect, an embodiment of the present disclosure provides a data transmission system, comprising: a cloud desktop server and a cloud desktop client; the cloud desktop server is configured to perform block processing on a dirty area in a cloud desktop video screen to be transmitted, obtain a target image block contained in the dirty area, determine multiple row hash values ​​and block hash values ​​of the target image block, wherein the multiple row hash values ​​correspond to multiple rows of pixels contained in the target image block, and the target image block is any one of the multiple image blocks obtained by the block processing; and, if it is determined that the target image block is stored in the cache space based on the block hash value, the multiple row hash values, the block cache table, and the row cache table, target cache index information of the target image block is determined, and pixel position information corresponding to the target image block in the cloud desktop video screen and the target cache index information are sent to the cloud desktop client, wherein the block cache table is configured to store block cache index information of each cached image block, and the row cache table is configured to store row cache index information of different rows of pixels in each cached image block; the cloud desktop client is configured to obtain the target image block from the cache space based on the target cache index information. The target image block is then displayed based on the pixel position information. The data transmission scheme provided by the embodiments of the present disclosure can be used in scenarios where a server transmits a video frame to a client. When the server needs to transmit a frame of video to the client, it first determines a dirty area in the frame of video. The dirty area is then divided into multiple image blocks, and the following processing is performed on each image block: For example, taking any one of the multiple image blocks as a target image block, the target image block contains multiple rows of pixels. First, multiple row hash values ​​corresponding to the multiple rows of pixels are calculated, and then a block hash value corresponding to the target image block is calculated.Next, based on the target image block's block hash value, multiple row hash values, and the configured block cache table and row cache table, a determination is made as to whether the target image block is stored in the cache space. If the target image block is stored, meaning it has already been cached, the target cache index information for the target image block is determined based on the block cache table and row cache table. This target cache index information indicates the storage address of the target image block in the cache space. The server then sends this target cache index information and the pixel position information corresponding to the target image block in the current video frame to the client. The client retrieves the target image block from the cache space based on the target cache index information and displays it at the corresponding position based on the pixel position information of the target image block. In other words, the client can maintain the pixels at other pixel positions in the previously displayed video frame and update only the pixels at the pixel position corresponding to the target image block with the pixels of the target image block read from the cache space, thereby obtaining the video frame that the server currently wants to transmit to the client for display. In the above solution, a two-level cache is provided: a block cache table and a row cache table. The block cache table is used to store block cache index information for cached image blocks, and the row cache table is used to store row cache index information for different rows of pixels in each cached image block. This two-level cache improves the cache hit rate, thereby reducing the amount of data required to be encoded by the server, as cached image content does not need to be repeatedly encoded and transmitted. This ultimately reduces the bandwidth required for video transmission on the server. BRIEF DESCRIPTION OF THE DRAWINGS To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly describes the drawings used in describing the embodiments. Obviously, the drawings described below represent some embodiments of the present disclosure. Those skilled in the art can derive other drawings based on these drawings without inventive effort. Figure 1 is a flowchart of a data transmission method provided by an embodiment of the present disclosure; Figure 2 is a schematic diagram of a two-level cache structure provided by an embodiment of the present disclosure; Figure 3 is a flowchart of a cache table query process provided by an embodiment of the present disclosure; Figure 4 is a schematic diagram of a situation in which an image block hits a row cache table provided by an embodiment of the present disclosure; Figure 5 is a flowchart of a data transmission method provided by an embodiment of the present disclosure; Figure 6 is a flowchart of a difference pixel encoding method provided by an embodiment of the present disclosure; Figure 7 is a flowchart of a cache table update method provided by an embodiment of the present disclosure; Figure 8 is a schematic diagram of a data transmission system provided by an embodiment of the present disclosure; Figure 9 is a structural schematic diagram of a data transmission device provided by an embodiment of the present disclosure; Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.DETAILED DESCRIPTION To further clarify the objectives, technical solutions, and advantages of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. It should be noted that the described embodiments represent only a portion of the embodiments of the present disclosure, and are not exhaustive. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without inventive effort are within the scope of protection of the present disclosure. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display) involved in the embodiments of the present disclosure are all authorized by the user or fully authorized by all parties. The collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or reject. The following detailed description of some embodiments of the present disclosure is provided in conjunction with the accompanying drawings. The following embodiments and features may be combined unless there is a conflict between the embodiments. Furthermore, the sequence of steps in the following method embodiments is provided for illustrative purposes only and is not intended to be a strict limitation. The data transmission solution provided by the embodiments of the present disclosure can be applied to scenarios where video data is transmitted between a server and a client. The server can be a cloud server, running various cloud applications such as cloud desktops and cloud games. Clients access these cloud applications by transmitting encoded video frames to the client in the form of a video stream, which the client decodes and displays. Figure 1 is a flow chart of a data transmission method provided by an embodiment of the present disclosure, which can be performed by a server. As shown in Figure 1, the method may include the following steps:

[0002] 101. The server performs block processing on a dirty area in a video image to be transmitted to obtain a target image block contained in the dirty area, where the target image block is any one of multiple image blocks obtained by the block processing.

[0003] 102. The server determines multiple row hash values ​​and block hash values ​​of a target image block, where the multiple row hash values ​​correspond to multiple rows of pixels included in the target image block.

[0004] 103. If the server determines that the target image block is stored in the cache space based on the block hash value, the multiple row hash values, the block cache table, and the row cache table, the server determines target cache index information of the target image block, where the block cache table is used to store block cache index information of each cached image block, and the row cache table is used to store row cache index information of different rows of pixels in each cached image block.

[0005] 104. The pixel position information corresponding to the target image block in the video image and the target cache index information are sent to the client, so that the client can retrieve the target image block from the cache space based on the target cache index information and display the target image block based on the pixel position information. Generally speaking, when transmitting a video stream from a server to a client, the server needs to encode each frame of the video stream using a compression coding algorithm, transmit the encoded video stream to the client, and perform the corresponding decoding to restore the video image for display, so that the user can see a smooth video image through the client. To save bandwidth required for transmission from the server to the client, the server does not fully encode each video frame. Instead, when transmitting a current video frame, the server first identifies the dirty areas of the video frame compared to the previous video frame, that is, the image areas that have changed, and encodes only these dirty areas. In this way, after receiving the encoding result of the dirty region, the client decodes the encoded dirty region and replaces the decoded dirty region with the corresponding image region in the previously displayed video frame, thereby obtaining the current video frame. Specifically, except for the pixel region of the dirty region, the remaining image regions of the current video frame are identical to those of the previous video frame. If the previous video frame is already displayed, the remaining image regions of the previous video frame, except for the target image region, remain unchanged. Only the decoded dirty region is replaced with the target image region to obtain the currently displayed video frame. The target image region is the image region that has changed between the previous and current video frames. While this dirty region encoding method can reduce the network bandwidth required for video streaming from the server to the client, if the dirty region is large and the video is updated frequently, the amount of data required to encode the dirty region is still relatively large, thus still occupying a significant amount of network bandwidth. To further reduce network bandwidth usage, in the disclosed embodiments, the server employs a two-level cache approach to cache image blocks and rows of pixels within video blocks already transmitted to the client. This improves cache hit rates, reduces the amount of data required to be encoded by the server, and thus reduces network bandwidth requirements. Specifically, the server first performs block processing on dirty regions within the video to be transmitted, obtaining multiple image blocks contained within the dirty regions. The dirty region identification method can be implemented using existing technologies and will not be detailed here. In practical applications, the server can determine the image block size as needed, for example, using pixel sizes of 64*64, 128*128, or other specifications.For example, the server can determine the image block size based on the client's screen resolution: the higher the screen resolution, the larger the image block size. Another example is that the server can determine the image block size based on computational efficiency requirements: when higher computational efficiency is required, a larger image block size is used; conversely, when lower computational efficiency is not required, a smaller image block size can be used. Since the processing procedures for the aforementioned multiple image blocks are the same, in this embodiment, any one of these image blocks is used as the target image block Y, and the processing procedure for the target image block Y is described as an example. For the target image block Y, multiple row hash values ​​and block hash values ​​corresponding to the target image block Y are calculated. The multiple row hash values ​​correspond to the multiple rows of pixels contained in the target image block Y. Assuming that the target image block Y is 64*64 in size, i.e., contains 64 rows of pixels, a row hash value is determined for each row of pixels, resulting in 64 row hash values ​​for the target image block Y. The block hash value is the hash value of the target image block Y. Optionally, the block hash value can be obtained by performing a hash calculation on multiple row hash values ​​of the target image block Y. For example, the 64 row hash values ​​in the above example are used as input to a hash algorithm, and the output result is the block hash value. In practical applications, the hash algorithms used for the above-mentioned row hash values ​​and block hash values ​​have the characteristic of "uniqueness", that is, different rows of pixels correspond to different hash values, and different image blocks correspond to different hash values. For example, hash algorithms such as the MD5 Message-Digest Algorithm (MD5) and the Secure Hash Algorithm (SHA) can be used. Based on the calculation of the above-mentioned block hash values ​​and row hash values, the embodiments of the present disclosure provide a two-level cache structure consisting of a block cache table and a row cache table. In practical applications, these two cache tables are maintained on the server side, while the cache space for data caching is located on the client side. It should be noted that the two cache tables share the same cache space, or in other words, the row cache table is established based on the block cache table, and no independent cache space is allocated for the row cache table. To facilitate understanding of the above-mentioned two-level cache structure, an example is provided in conjunction with Figure 2. For convenience of description, Let's take any image block X in the first frame of a video stream as an example. It's understandable that the entire first frame can be considered a dirty area. By dividing it into a set image block size (assuming 64*64), multiple image blocks can be obtained. Assume that image block X is the first image block traversed among these.In this initial state, the block cache table and row cache table are empty, and the cache space is empty. Based on the above assumptions, the row hash values ​​corresponding to each of the 64 rows of pixels in image block X are calculated, thereby obtaining 64 row hash values, such as H1, H2, ..., H64. Furthermore, assume that the block hash value K1 corresponding to image block X is calculated based on these 64 row hash values. Subsequently, the server queries the block cache table and finds that the block hash value K1 does not exist in the block cache table (because the block cache table is empty at this time). Therefore, the server constructs a table entry B1 corresponding to the block hash value K1 in the block cache table. To construct this table entry B1, the server first needs to allocate block cache index information for image block X. This block cache index information indicates the storage address of image block X in the cache space, such as address L1 shown in Figure 2. Since each image block has the same size and the cache space is used to store data in units of image blocks, the memory address length occupied by each image block in the cache space is fixed and continuous. Therefore, the server can determine the memory address corresponding to the current image block X based on the number of block hash values ​​stored in the block cache table and the memory addresses corresponding to each block hash value. As shown in Figure 2, the server constructs a table entry B1 in the block cache table containing the block hash value K1 and the block cache index information (i.e., the memory address L1 of image block X). In practical applications, this table entry B1 may also include the aforementioned 64 row hash values: H1, H2, H4, H5, H64. OIn addition, optionally, the table entry B1 may also include the perceptual hash value G1 corresponding to the image block X mentioned below. As mentioned above, the row cache table is generated based on the block cache table. Therefore, after obtaining the table entry B1 in the block cache table, multiple table entries corresponding to the image block X can be constructed in the row cache table: C1, C2, ..., C64, corresponding to the above 64 row hash values ​​respectively. Assuming that the server has not found the above 64 row hash values ​​in the row cache table at this time, 64 table entries corresponding to the 64 row hash values ​​can be constructed. Taking the table entry Ci corresponding to one of the row hash values ​​Hi as an example, the table entry Ci includes the row hash value Hi corresponding to the i-th row of pixels in the image block X and its row cache index information. The row cache index information includes the block cache index information corresponding to the image block X (that is, the storage address L1 of the image block X) and the row number io corresponding to the i-th row of pixels in the image block X. The row number i refers to the row number of the row of pixels corresponding to the row hash value Hi in the image block X to which it belongs. It is a relative row number, not the row number in a complete frame of video. It is understood that the hash values ​​for each row in the row cache table and the block cache index information corresponding to these row hash values ​​are all derived from the table entry B1 corresponding to image block X in the block cache table. As shown in the example of FIG2 , the block cache table is used to store the block cache index information for each cached image block, and the row cache table is used to store the row cache index information for pixels in different rows within each cached image block. Furthermore, in the embodiment shown in FIG2 , since image block X does not match either the block cache table or the row cache table, the image block can be encoded and sent to the client. After decoding image block X, the client not only displays it on the screen but also stores it in the cache. Based on this, the message sent by the server to the client can include the encoded image block X and the block cache index information corresponding to image block X, so that the client stores image block X row by row in storage address L1 based on the block cache index information, with each pixel occupying a set byte length within storage address L1. It is understood that storage address L1 here should be understood as a segment address. Following the above processing process of the target image block Y, after obtaining multiple row hash values ​​and block hash values ​​of the target image block Y, it is possible to determine whether the target image block Y is already stored in the cache space based on the multiple row hash values, block hash values ​​of the target image block Y and the generated block cache table and row cache table. oIn simple terms, if an entry corresponding to the block hash value of the target image block Y is found in the block cache table, or if there is no entry corresponding to the block hash value in the block cache table but entries corresponding to multiple row hash values ​​of the target image block Y are found in the row cache table, it can be determined that the target image block Yo has been cached in the cache space. If it is determined that the target image block Ya has been stored in the cache space, the server can determine the target cache index information of the target image block Ya and send the pixel position information corresponding to the target image block Y in the current video screen and the target cache index information to the client. The pixel position information corresponding to the target image block Y in the current video frame refers to the position coordinates corresponding to the target image block Y in the video frame, which can be represented by the pixel coordinates of its upper left corner and lower right corner. It is understood that if an entry corresponding to the block hash value of the target image block Y is found in the block cache table, then the target cache index information is the block cache index information corresponding to the target image block Y in the block cache table—the storage address corresponding to the target image block Y in the cache space. If an entry corresponding to multiple row hash values ​​of the target image block Y is found in the row cache table, then the target cache index information is the row cache index information corresponding to the multiple row hash values ​​of the target image block Y—the storage address corresponding to each row hash value in the cache space and the row number corresponding to each row hash value in the corresponding image block. In summary, after obtaining the pixel position information corresponding to the target image block Y in the current video frame and the target cache index information, the client retrieves the target image block Y from the cache space based on the target cache index information and displays the target image block Y based on the pixel position information. It should be noted that if the block hash value corresponding to the target image block Y is found in the block cache table, the target image block Y can be directly read from the cache space based on the block cache index information corresponding to the block hash value. However, if the row hash values ​​corresponding to the target image block Y are found in the row cache table, the client reads each row of pixels of the target image block Y from the cache space based on the row cache index information corresponding to each row hash value. The pixels are then displayed row by row at the pixel positions corresponding to the aforementioned pixel position information, based on the row numbers corresponding to the row hash values. FIG3 is a flowchart of a cache table query process provided in an embodiment of the present disclosure. As shown in FIG3 , the query process includes the following steps:

[0006] 301. The server checks the block cache table to see if it finds the first entry containing the block hash value of the target image block Y. If so, execute steps 302-303; otherwise, execute step 304. 302. The server determines that the target cache index information of the target image block Y is the block cache index information corresponding to the block hash value contained in the first entry.

[0007] 303. The server sends the pixel position information corresponding to the target image block Y in the video image and the block cache index information of the target image block Y to the client. In this embodiment, the target image block Y is still used as an example. When querying the cache table, the block cache table is first queried. If the first table entry containing the block hash value of the target image block Y is found in the block cache table based on the block hash value of the target image block Y, it indicates that the target image block Y is stored in the cache space. o The block cache index information stored in the first table entry indicates the storage address of the target image block Y in the cache space. The image block read from this storage address is the target image block Y. The client then locates the pixel position of the target image block Y in the display area of ​​the video screen and renders the target image block Y. It should be understood that the target image block Y already being stored in the cache space here should be understood as meaning that the target image block Y has appeared in other video screens and has been cached, and has just appeared in the current video screen.

[0008] 304. The server searches the row cache table to see whether multiple second entries corresponding to multiple row hash values ​​of the target image block Y are found. If so, execute steps 305-306; otherwise, execute step 307.

[0009] 305. The server obtains row cache index information corresponding to the target row hash value from the target second table entry containing the target row hash value.

[0010] 306. The server sends the pixel position information corresponding to the target image block Y in the video image and the row cache index information corresponding to each of the multiple row hash values ​​of the target image block Y to the client. When querying the row cache table, if the second table entry corresponding to each of the multiple row hash values ​​of the target image block Y is found in the row cache table, it indicates that the target image block Yo is stored in the cache space. For any target row hash value among the multiple row hash values, the server obtains the row cache index information corresponding to the target row hash value from the target second table entry containing the target row hash value. The target row hash value corresponds to a target row pixel in the target image block Y. The row cache index information corresponding to the target row hash value includes block cache index information and row number information corresponding to the target row hash value. The block cache index information indicates the storage address of the image block containing the target row pixel in the cache space, and the row number information indicates the row number corresponding to the target row pixel in the corresponding image block. For ease of understanding, FIG. 4 is used to illustrate a case where the block hash value of the target image block Y is not found in the block cache table, but multiple row hash values ​​of the target image block Y are found in the row cache table. As shown in Figure 4, assume that the block cache table previously stores block hash values ​​for image blocks Z1 and Z2. Furthermore, assume that the row hash values ​​for image block Z1 include Z1H1, Z1H2, and Z1H64, which correspond sequentially to its 64 rows of pixels, and that the row hash values ​​for image block Z2 include Z2H1, Z2H2, and Z2H64, which correspond sequentially to its 64 rows of pixels. The row hash values ​​corresponding to the 64 rows of pixels of the current target image block Y are Z1H32, Z1H33, Z1H64, Z2H1, Z2H2, and Z2H32, respectively. In other words, assume that the target image block Y is composed of the lower half of image block Z1 and the upper half of image block Z2. Because the target image block Y differs from the pixels of image blocks Z1 and Z2, the block hash value of the target image block Y differs from the block hash values ​​of both image blocks Z1 and Z2. In the above example scenario, 64 second table entries Z1H32, Z1H33, Z1H64, Z2H1, Z2H2, and Z2H32 can be hit in the row cache table. Among them, the "block cache index information" stored in the 32 second table entries corresponding to Z1H32, Z1H33, and Z1H64 are all the storage address L2 corresponding to the image block Z1 in the cache space, and the "block cache index information" stored in the 32 second table entries corresponding to Z2H1, Z2H2, ..., and Z2H32 are all the storage address L3 corresponding to the image block Z2 in the cache space.Based on this, the client reads the 32nd to 64th rows of pixels of image block Z1 at storage address L2 according to the row numbers 32-64 as the upper 32 rows of pixels of target image block Y. At storage address L3, the client reads the 1st to 32nd rows of pixels of image block Z2 according to the row numbers 1-32 as the lower 32 rows of pixels of target image block Y, thereby obtaining the 64 rows of pixels contained in the target image block Y. The client then renders these 64 rows of pixels within the pixel position area corresponding to the target image block Y.

[0011] 307. The server encodes the target image block Y and sends the encoded target image block Y and the pixel position information corresponding to the target image block Y in the video image to the client. In this embodiment, if neither the block hash value nor each row hash value of the target image block Y matches in either the block cache table or the row cache table, it is determined that the target image block Y is not cached in the cache space. Optionally, the target image block Y can be directly encoded, and the encoded target image block Y and the pixel position information corresponding to the target image block Y in the video image can be sent to the client. After decoding the target image block Y, the client renders and displays it based on the corresponding pixel position information. In the above solution, a two-level cache is provided: a block cache table and a row cache table. The block cache table is used to store block cache index information for each cached image block, and the row cache table is used to store row cache index information for different rows of pixels in each cached image block. This two-level cache improves the cache hit rate, thereby reducing the amount of data required to be encoded by the server, and ultimately reducing the bandwidth required for transmitting the video image on the server. FIG5 is a flowchart of a data transmission method provided by an embodiment of the present disclosure. As shown in FIG5 , the method may include the following steps:

[0012] 501. The server performs block processing on the dirty area in the video image to be transmitted to obtain the target image block contained in the dirty area.

[0013] 502. The server determines multiple row hash values ​​and block hash values ​​of the target image block, where the multiple row hash values ​​correspond to multiple rows of pixels included in the target image block.

[0014] 503. If the server determines that the target image block is not stored in the cache space according to the block hash value, the multiple row hash values, the block cache table, and the row cache table, the server determines a perceptual hash value of the target image block.

[0015] 504. If the server finds a similar image block having a perceptual hash value similar to the perceptual hash value of the target image block in the block cache table, the server obtains block cache index information of the similar image block contained in the block cache table.

[0016] 505. The server determines the difference pixels between the similar image block and the target image block, and encodes the difference pixels.

[0017] 506. The server sends the block cache index information of the similar image blocks, the encoded difference pixels, and the corresponding pixel position information of the target image block in the video image to the client. In this embodiment, to further reduce the network bandwidth required for video transmission from the server to the client, similar image blocks are found using the perceptual hash value of the target image block. Only the difference pixel values ​​between the target image block and the similar image blocks are encoded. This can further reduce the amount of image encoding data and bandwidth usage. Unlike the previously described block hash values ​​and row hash values, which have the "uniqueness" characteristic, perceptual hash values ​​are not "unique." That is, if some pixels in two image blocks differ, the block hash values ​​of the two image blocks will be different, but the perceptual hash values ​​of the two image blocks will have a similarity exceeding a set threshold (e.g., 92%). Therefore, the hash algorithm used to calculate the block hash values ​​and row hash values ​​previously described is different from the hash algorithm used here for perceptual hash values. The similarity of the perceptual hash values ​​of two image blocks can be determined based on the Hamming distance between their perceptual hash values. As shown in FIG2 , an entry in the block cache table includes not only the block hash value of the corresponding image block but also the perceptual hash value of the image block. When the server misses a hit in the block cache table entry based on the block hash value of the target image block and misses a hit in the row cache table based on each row hash value of the target image block, it calculates the perceptual hash value of the target image block and queries the block cache table to see if it finds a hit in the entry corresponding to the perceptual hash value. If the similarity between the perceptual hash value corresponding to image block P in the block cache table and the perceptual hash value of the target image block exceeds a set threshold, it is determined that image block P is similar to the target image block and the block cache index information (indicating the storage address of image block P in the cache space) of image block P is obtained from the block cache table. Furthermore, the server determines the difference pixels between image block P and the target image block and encodes the difference pixels. The server can compare image block P with the target image block pixel by pixel to obtain pixels with different pixel values ​​at two corresponding positions as the difference pixels. It is understood that the number of difference pixels between the two image blocks described above is often multiple and may be discretely distributed. Therefore, when encoding the difference pixels, the smallest rectangular image region containing all the difference pixels can be determined and encoded. The server then sends the block cache index information of the similar image block P, the encoded difference pixels, and the corresponding pixel position information of the target image block in the video image to the client.The client retrieves the similar image block P from the cache space based on its block cache index information, displays it based on the pixel position information of the target image block, and then updates the decoded difference pixels to the corresponding pixel positions in the similar image block P. Simply put, the client first renders and displays the similar image block P at the pixel position area where the target image block needs to be displayed. It then locates the decoded difference pixels in this pixel position area and replaces the pixel values. Therefore, by searching for similar image blocks using perceptual hash values, only the difference pixels can be encoded and transmitted, further reducing network bandwidth usage. It is understood that if no similar image block is found based on the perceptual hash value of the target image block, the target image block must be encoded and transmitted. A method for encoding difference pixels is provided below in conjunction with Figure 6. Figure 6 is a flowchart of a method for encoding difference pixels provided in an embodiment of the present disclosure. As shown in Figure 6, the method may include the following steps:

[0018] 601. Determine an image block type corresponding to a target image block.

[0019] 602. Determine an encoding method corresponding to the image block type.

[0020] 603. Encode the difference pixels using the above encoding method. In an optional embodiment, the target image block can be identified based on a set classification neural network model. Image block types include text and image. Simply put, if the target image block contains mostly text, the target image block is determined to be a text-type image block. Conversely, if the target image block contains mostly images, the target image block is determined to be a picture-type image block. In another optional embodiment, the image block type corresponding to the target image block may be determined in the following manner: obtaining a threshold value for the total pixel number ratio of a first color number N1, a second color number N2, and a third color number N3, where the first color number N1 is less than the second color number N2, and the third color number N3 is less than the second color number threshold N2; determining the number of colors contained in the target image block and the pixel number ratio of each color; if the number of colors contained in the target image block is less than the first color number N1, determining the target image block to be a text type; if the number of colors contained in the target image block is between the first color number N1 and the second color number N2, and the total pixel number ratio corresponding to the third color number with the highest pixel number ratio in the target image block is greater than the total pixel number ratio threshold value, determining the target image block to be a text type; if the number of colors contained in the target image block is between the first color number N1 and the second color number, and the total pixel number ratio corresponding to the third color number with the highest pixel number ratio in the target image block is less than the total pixel number ratio threshold value, determining the target image block to be a picture type; if the number of colors contained in the target image block is greater than the second color number N2, The target image block is then determined to be of picture type. In practical applications, the number of colors in a text-type image block is often lower than that in a picture-type image block. Therefore, a first color number N1 and a second color number N2 are set. If the number of colors in the target image block is lower than N1, the target image block can be directly determined to be of text type. If the number of colors in the target image block is greater than N2, the target image block can be directly determined to be of picture type. If the number of colors in the target image block is greater than N1 but less than N2, the type of the target image block needs to be further determined based on the color distribution in the target image block. The color distribution mainly determines whether the colors are relatively concentrated. If the colors are relatively concentrated, the target image block is determined to be of text type; otherwise, the target image block is determined to be of picture type.The process for determining whether colors are concentrated is as follows: The number of pixels occupied by each color in the target image block is determined, thereby obtaining the pixel percentage of each color in the target image block. Assuming that the threshold for the total pixel percentage of the third color number N3 is set to 80%, the total pixel percentage of the top N3 colors in the target image block (i.e., the cumulative sum of the pixel percentages of these N3 colors) is calculated. If the total pixel percentage of these N3 colors is greater than the threshold of 80%, the target image block is determined to be a text-type image block; otherwise, it is an image-type image block. In practical applications, each color corresponds to a different RGB pixel value, so calculating the pixel percentage of each color is equivalent to calculating the pixel percentage of each RGB pixel value, that is, the number of occurrences of each RGB pixel value in the target image block. If the target image block is determined to be a text-type image block, lossless encoding can be used for encoding the target image block; if the target image block is determined to be an image-type image block, lossy encoding can be used for encoding the target image block. Subsequently, based on the determined encoding method, the difference pixel region in the target image block is encoded. It should be noted that, as described above, when encoding of the target image block is determined based on the perceptual hash value, block hash value, or row hash value of the target image block, the image block type corresponding to the target image block may also be determined first, and the entire target image block may be encoded based on the encoding method corresponding to the image block type. FIG7 is a flowchart of a cache table updating method provided in an embodiment of the present disclosure. As shown in FIG7 , the method may include the following steps: 701. If the server determines that the target image block is not stored in the cache space based on the block hash value, multiple row hash values, the block cache table, and the row cache table of the target image block, the server determines the perceptual hash value of the target image block.

[0021] 702. If the server finds a similar image block having a perceptual hash value similar to the perceptual hash value of the target image block in the block cache table, the server obtains block cache index information of the similar image block contained in the block cache table, determines difference pixels between the similar image block and the target image block, and encodes the difference pixels.

[0022] 703. The server determines the block cache index information of the target image block based on the block cache index information of each image block stored in the block cache table, and adds a third table entry including the block cache index information, block hash value, and perceptual hash value of the target image block to the block cache table.

[0023] 704. The server sends the block cache index information of the target image block, the block cache index information of the similar image block, the encoded difference pixel, and the pixel position information corresponding to the target image block in the video picture to the client, so that the client stores the target image block generated according to the similar image block and the difference pixel in the cache space according to the block cache index information of the target image block.

[0024] 705. The server adds multiple fourth entries to the row cache table. Each target fourth entry includes the block cache index information of the target image block, a target row hash value, and the row number corresponding to the target row hash value. The target row hash value is any one of the multiple row hash values. In this embodiment, if the server does not find a match between the block hash value of the target image block in the block cache table and all the row hash values ​​of the target image block in the row cache table, it calculates the perceptual hash value of the target image block and queries the block cache table to see if a similar image block with a perceptual hash value that meets a similarity threshold is found. If a similar image block is found, as described above, the block cache index information of the similar image block, the encoded difference pixels, and the pixel position information of the target image block in the current video image can be sent to the client, allowing the client to restore the target image block. Furthermore, the server also needs to update the block cache table and the row cache table. Specifically, the block cache table is updated as follows: the server determines the block cache index information of the target image block based on the block cache index information of each image block stored in the block cache table, and adds a third table entry containing the block cache index information, block hash value, and perceptual hash value of the target image block to the block cache table. Determining the block cache index information of the target image block based on the block cache index information of each image block stored in the block cache table effectively determines that the storage address of the target image block in the cache space is the next segment of storage addresses based on the storage address of the last image block cached in the block cache table. Therefore, the message sent by the server to the client also needs to include the block cache index information of the target image block. In this way, after the client restores the target image block based on the similar image blocks and the difference pixels, it stores the restored target image block in the cache space according to the storage address indicated in the block cache index information of the target image block. Restoring the target image block involves replacing the difference pixels with corresponding pixel positions in the similar image blocks. The row cache table is updated by adding a fourth entry corresponding to each row hash value of the target image block to the row cache table. The fourth entry corresponding to the target row hash value includes the block cache index information, the target row hash value, and the row number information corresponding to the target row hash value for the target image block. In an optional embodiment, referring to FIG. 2 , the third entry may also include multiple row hash values ​​for the target image block. Based on this, when it is determined that cache information for the target image block needs to be deleted, the third entry is deleted, and multiple fourth entries are deleted based on the multiple row hash values ​​included in the third entry.In practical applications, the cache space has an upper limit on storage capacity. When the cache space is full or reaches a set threshold, previously stored image blocks must be deleted in a first-in, first-out order. Consequently, the entries corresponding to the deleted image blocks in the block cache and row cache tables must also be deleted. The deletion process involves first locating the corresponding entry in the block cache table based on the block hash value of the deleted image block. Then, based on the row hash value contained in the block cache entry, the corresponding entry in the row cache table is located. Both the identified entry in the block cache and the identified entry in the row cache table are deleted. Finally, it should be noted that if no similar image block is found in the block cache table based on the perceptual hash value of the target image block, the target image block is encoded in its entirety and sent to the client. Furthermore, the block cache and row cache tables must be updated. The update process is similar to the above process, except that the client recovers the target image block by decoding the encoded target image block. The video data transmission methods described in the embodiments of the present disclosure are applicable to a variety of application scenarios requiring video data transmission, including but not limited to cloud desktop and live streaming scenarios. As shown in FIG8 , in a cloud desktop application scenario, the video data transmission system includes the cloud desktop server and cloud desktop client shown in the figure. The cloud desktop server is configured to perform block processing on dirty areas within the cloud desktop video image to be transmitted, obtain a target image block contained in the dirty area (the target image block is any one of the multiple image blocks obtained by the block processing), determine multiple row hash values ​​(the multiple row hash values ​​correspond to multiple rows of pixels contained in the target image block) and a block hash value for the target image block, and if, based on the block hash value, the multiple row hash values, the block cache table, and the row cache table, determine that the target image block is stored in the cache space, determine target cache index information for the target image block, and send the pixel position information and target cache index information corresponding to the target image block in the cloud desktop video image to the cloud desktop client. The block cache table is configured to store the block cache index information for each cached image block, and the row cache table is configured to store the row cache index information for different rows of pixels in each cached image block. The cloud desktop client is configured to retrieve the target image block from the cache space based on the target cache index information and display the target image block based on the pixel position information. The specific processing procedures of the cloud desktop server and client can be found in the relevant descriptions of the aforementioned other embodiments and are not further elaborated here. The following describes in detail one or more data transmission devices according to embodiments of the present disclosure. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components and the steps described in this solution.FIG9 is a schematic structural diagram of a data transmission device provided by an embodiment of the present disclosure. The data transmission device is applied to a server. As shown in FIG9 , the device includes: a blocking module 11, a determination module 12, a query module 13, and a sending module 14. The blocking module 11 is configured to perform blocking processing on dirty areas in a video image to be transmitted, and obtain a target image block contained in the dirty area. The target image block is any one of multiple image blocks obtained by the blocking processing. The determination module 12 is configured to determine multiple row hash values ​​and block hash values ​​for the target image block, wherein the multiple row hash values ​​correspond to multiple rows of pixels contained in the target image block. The query module 13 is configured to determine target cache index information for the target image block if it is determined that the target image block is stored in the cache space based on the block hash value, the multiple row hash values, the block cache table, and the row cache table. The block cache table is configured to store block cache index information for each cached image block, and the row cache table is configured to store row cache index information for different rows of pixels in each cached image block. The sending module 14 is configured to send the pixel position information corresponding to the target image block in the video image and the target cache index information to the client, so that the client retrieves the target image block from the cache space based on the target cache index information and displays the target image block based on the pixel position information. The apparatus shown in FIG9 can perform the steps in the aforementioned embodiments. The detailed execution process and technical effects are described in the aforementioned embodiments and are not repeated here. In one possible design, the structure of the apparatus shown in FIG9 can be implemented as an electronic device. As shown in FIG10 , the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable code. When the executable code is executed by the processor 21, the processor 21 can at least implement the data transmission method provided in the aforementioned embodiments. Furthermore, embodiments of the present disclosure provide a non-transitory machine-readable storage medium. The non-transitory machine-readable storage medium stores executable code. When the executable code is executed by the processor of the electronic device, the processor can at least implement the data transmission method provided in the aforementioned embodiments. The device embodiments described above are merely illustrative. The network elements described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the solutions in this embodiment. Persons of ordinary skill in the art can understand and implement the present invention without inventive effort.Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding the necessary general-purpose hardware platform, or of course, by a combination of hardware and software. Based on this understanding, the essence of the above technical solutions, or the portion that contributes to the prior art, can be embodied in the form of a computer product. The present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present disclosure, and are not intended to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art will understand that the technical solutions described in the aforementioned embodiments may be modified, or some of the technical features thereof may be replaced by equivalents. Such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

Claims 1. A data transmission method, applied to a server, comprising: Performing block processing on a dirty area in the video picture to be transmitted to obtain a target image block contained in the dirty area, where the target image block is any one of the multiple image blocks obtained by the block processing; Determine a plurality of row hash values ​​and a block hash value of the target image block, wherein the plurality of row hash values ​​correspond to a plurality of rows of pixels included in the target image block, and the block hash value is a hash value corresponding to the target image block; If it is determined that the target image block is stored in the cache space according to the block hash value, the multiple row hash values, the block cache table, and the row cache table, target cache index information of the target image block is determined, wherein the block cache table is used to store block cache index information of each cached image block, and the row cache table is used to store row cache index information of pixels in the same row in each cached image block; The pixel position information corresponding to the target image block in the video picture and the target cache index information are sent to the client, so that the client obtains the target image from the cache space according to the target cache index information and displays the target image block according to the pixel position information.

2. The method according to claim 1, wherein: If it is determined that the target image block is stored in the cache space based on the block hash value, the multiple row hash values, the cache table, and the row cache table, then determining the destination cache index information of the target image block includes: if a first table entry corresponding to the block hash value is queried in the block cache table, then determining that the target image block is stored in the cache space; and determining that the destination cache index information of the target image block is the cache index information corresponding to the block hash value contained in the first table entry, the block cache index information indicating a storage address of the target image block in the cache space.

3. The method according to claim 1, wherein: If it is determined that the target image block is stored in the cache space based on the block hash value, the multiple row hash values, the cache table, and the row cache table, then determining target cache index information of the target image block includes: if a first table entry containing the block hash value is not found in the block cache table, querying the multiple row hash values ​​in the row cache table; if multiple second table entries corresponding to the multiple row hash values ​​are found in the row cache table, then determining that the target image block is stored in the cache space; for a target row hash value, obtaining row cache index information corresponding to the target row hash value in a target second table entry containing the target row hash value, wherein the target row hash value is any one of the multiple row hash values, and the target row hash value corresponds to a target row of pixels; wherein the row cache index information includes block cache index information and row number information corresponding to the target row hash value, the block cache index information indicating a storage address of the image block containing the target row of pixels in the cache space, and the row number information indicating a row number corresponding to the target row of pixels in the corresponding image block; and determining the target cache index information of the target image block as follows: The row cache index information corresponding to each of the multiple row hash values.

4. The method according to claim 1, wherein: The method further comprises: If it is determined, based on the block hash value, the multiple row hash values, the block cache table, and the row cache table, that the target image block is not stored in the cache space, determining a perceptual hash value of the target image block; if a similar image block having a perceptual hash value similar to the perceptual hash value of the target image block is found in the block cache table, obtaining block cache index information of the similar image block contained in the block cache table; determining difference pixels between the similar image block and the target image block; encoding the difference pixels; and sending the block cache index information of the similar image block, the encoded difference pixels, and pixel position information corresponding to the target image block in the video picture to a client, so that the client obtains the similar image block from the cache space according to the block cache index information of the similar image block, displays the similar image block according to the pixel position information, and updates the decoded difference pixels to corresponding pixel positions in the similar image block.

5. The method according to claim 4, wherein: The encoding of the difference pixels includes: determining an image block type corresponding to the target image block; determining an encoding method corresponding to the image block type; and encoding the difference pixels using the encoding method.

6. The method according to claim 5, wherein: Determining the image block type corresponding to the target image block includes: obtaining a set threshold value for a total pixel number ratio of a first color number, a second color number, and a third color number, wherein the first color number is less than the second color number, and the third color number is less than the second color number; determining the number of colors contained in the target image block and a pixel number ratio of each color; if the number of colors contained in the target image block is less than the first color number, determining the target image block to be a text type; if the number of colors contained in the target image block is between the first color number and the second color number, and the total pixel number ratio corresponding to the third color number with the highest pixel number ratio in the target image block is greater than the total pixel number ratio threshold value, determining the target image block to be a text type; if the number of colors contained in the target image block is between the first color number and the second color number, and the total pixel number ratio corresponding to the third color number with the highest pixel number ratio in the target image block is less than the total pixel number ratio threshold value, determining the target image block to be a picture type; and if the number of colors contained in the target image block is greater than the second color number, determining the target image block to be a picture type.

7. The method according to claim 6, wherein: The encoding method corresponding to the text type is a lossless encoding method, and the encoding method corresponding to the image type is a lossy encoding method.

8. The method according to claim 4, wherein: The method further includes: determining block cache index information of the target image block based on block cache index information of each image block stored in the block cache table; adding a third table entry including the block cache index information of the target image block, the block hash value, and the sensitive hash value to the block cache table; 17. Sending the block cache index information of the target image block to the client, so that the client stores the target image block generated according to the similar image block and the difference pixel in the cache space according to the block cache index information of the target image block; adding a plurality of fourth table entries to the row cache table, wherein the target fourth table entries include the block cache index information of the target image block, a target row hash value, and row number information corresponding to the target row hash value, and the target row hash value is any one of the plurality of row hash values.

9. The method according to claim 8, wherein: The third table entry also includes the multiple row hash values.

10. The method according to claim 9, wherein: The method further includes: if it is determined that the cache information of the target image block needs to be deleted, deleting the third entry, and deleting the plurality of fourth entries according to the plurality of row hash values ​​included in the third entry.

11. The method according to claim 1, wherein: The method further includes: caching the target image block and the row of pixels in the target image block for the video picture that has been transmitted to the client in a two-level cache manner.

12. An electronic device, comprising: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the data transmission method according to one of claims 1 to 10.

13. A non-transitory machine-readable storage medium having executable code stored thereon, wherein when the executable code is executed by a processor of an electronic device, the processor is caused to execute the data transmission method according to any one of claims 1 to 10.

14. A computer program product comprising: A computer program, which, when executed by a processor of an electronic device, causes the processor to perform the data transmission method according to any one of claims 1 to 10.

15. A data transmission system, comprising: Cloud desktop server and cloud desktop client; The cloud desktop server is configured to perform block processing on a dirty area in a cloud desktop video image to be transmitted, obtain a target image block contained in the dirty area, and determine multiple row hash values ​​and block hash values ​​of the target image block, where the multiple hash values ​​correspond to multiple rows of pixels contained in the target image block, and the target image block is any one of the multiple image blocks obtained by the block processing; and, if the target image block is stored in a certain cache space according to the block hash value, the multiple row hash values, the block cache table, and the row cache table, determining target cache index information of the target image block, and sending pixel position information corresponding to the target image block in the cloud desktop video screen and the target cache index information to the cloud desktop client, wherein the block cache table is used to store block cache index information of each cached image block, and the row cache table is used to store row cache index information of different rows of pixels in each cached image block; The cloud desktop client is configured to obtain the target image block from the cache space according to the target cache index information, and display the target image block according to the pixel position information.

Images