Image processing method and terminal device, server

Abstract

This disclosure provides an image processing method and apparatus, relating to the field of image processing, which can solve the problem of large data volume in encoded desktop images. The specific technical solution is as follows: acquiring a desktop image, the desktop image comprising multiple image units; determining the type of each image unit based on its content; determining multiple encoding units of various sizes in the desktop image based on the type of each image unit, wherein each encoding unit comprises at least one image unit, and the maximum number of image unit types in each encoding unit is greater than or equal to a preset number; encoding the desktop image according to the multiple encoding units. This invention is used to reduce the data volume of encoded desktop images.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image processing method, terminal device, and server. Background Technology

[0002] In a cloud service system, the server can encode the desktop image generated by the virtual machine (VM) and send it to the terminal device over the network. The terminal device then decodes and displays the virtual desktop image.

[0003] Servers typically use 16-pixel x 16-pixel macroblocks for encoding to generate a bitstream, which is then sent to the terminal device. While dividing and encoding desktop images into fixed-size macroblocks is applicable to various image scenarios (both natural and computer-generated images) and has a simple algorithm implementation, the encoded desktop image data volume is still large, resulting in a long encoding and decoding time. Summary of the Invention

[0004] This disclosure provides an image processing method and apparatus that can solve the problem of large data volume in encoded desktop images. The technical solution is as follows:

[0005] According to a first aspect of the present disclosure, an image processing method is provided, the method comprising: acquiring a desktop image, the desktop image including a plurality of image units; determining a type of each image unit based on the content of each image unit; determining a plurality of encoding units of various sizes in the desktop image based on the type of each image unit, wherein the number of the most common type of image units in each encoding unit is greater than or equal to a preset number; and encoding the desktop image according to the plurality of encoding units.

[0006] It should be understood that this method can be applied to servers.

[0007] Encoding a desktop image using coding units of various sizes, with each coding unit including at least one unit of the image, can reduce the number of coding units in the desktop image, thereby reducing redundant information in the encoded data.

[0008] The type of each image unit is determined based on its image, and the encoding unit of the desktop image is determined based on the type of each image unit. This makes the determination of the encoding unit more in line with the different needs of each area of ​​the desktop image, and improves the accuracy between the encoded desktop image and the original desktop image.

[0009] In some embodiments, the translation vectors of each image unit in the plurality of image units of the first coding unit are equal, and the translation vector of each image unit is used to represent the positional relationship between the image unit and the image unit, wherein the reference unit image is the same as the image unit in the reference image, and the preset number is the number of image units in the first coding unit.

[0010] It should be understood that the reference image can be the desktop image or other images.

[0011] In the first coding unit, each image unit has the same positional relationship as the same image in the reference image. The image in the entire coding unit can be described with only one positional information to indicate the positional relationship, thereby reducing the number of coding units in the desktop image and reducing the amount of redundant information between coding units.

[0012] In some embodiments, the method further includes: sending an encoded desktop image to a terminal device, the encoded desktop image including encoding unit indication information, the encoding unit indication information implicitly indicating the position of an encoding unit that includes one of the image units among the plurality of encoding units.

[0013] Encoding unit indication information can implicitly indicate the position of an encoding unit that includes only one image unit in the desktop image, thereby reducing the amount of data in the encoded desktop image.

[0014] Specifically, the coding unit indication information includes the position of the coding unit that includes the plurality of image units among the plurality of coding units. The coding unit indication information can also be used to indicate the size of the coding unit that includes the plurality of image units.

[0015] Furthermore, the coding unit indication information may also include the type of each coding unit, wherein the type of each coding unit is the type of the most numerous image unit among the coding units.

[0016] In some embodiments, the method further includes: encrypting the encoding unit indication information, wherein the encoded desktop image includes the encrypted encoding unit indication information.

[0017] The encoding unit indication information is essential for the terminal device to recover the desktop image from the encoded desktop image, and its data volume is relatively small. Encrypting the encoding unit indication information improves the security of transmitting the encoded desktop image while reducing the amount of data that needs to be encrypted, thus increasing the encoding speed. The encrypted data contains a significant amount of redundant data; reducing the amount of data requiring encryption can decrease the consumption of transmission resources.

[0018] In one embodiment, when the color gradient of the image unit is greater than or equal to a preset gradient, the image unit is of a sharp type; when the color gradient of the image unit is less than the preset gradient, the image unit is of a gentle type.

[0019] The encoding unit is determined based on the color gradient of the image unit. This makes the encoding method used when encoding the image unit more consistent with the color gradient of each image unit in the encoding unit, thereby improving the accuracy between the encoded desktop image and the original desktop image.

[0020] Secondly, an image processing apparatus is provided, the transpose including a memory and a processor. The memory stores a program. When the program is executed in the processor, the processor is configured to: acquire a desktop image, the desktop image including a plurality of image units; determine a type of each image unit based on the image of each image unit; determine a plurality of encoding units of various sizes in the desktop image based on the type of each image unit, wherein each encoding unit includes at least one image unit, and the maximum number of image units of a certain type in each encoding unit is greater than or equal to a preset number; and encode the desktop image according to the plurality of encoding units.

[0021] Encoding a desktop image using coding units of various sizes, with each coding unit including at least one unit of the image, can reduce the number of coding units in the desktop image, thereby reducing redundant information in the encoded data.

[0022] In one embodiment, the translation vector of each image unit in the plurality of image units of the first coding unit is equal, and the translation vector of each image unit is used to represent the positional relationship between the image unit and the image unit, wherein the reference unit image is the same as the image unit in the reference image, and the preset number is the number of image units in the first coding unit.

[0023] It should be understood that the reference image can be the desktop image or other images.

[0024] The positional relationship between each image unit in the coding unit and the same image in the reference image is the same. The image in the entire coding unit can be described with only one positional information to indicate the positional relationship, thereby reducing the number of coding units in the desktop image and reducing the amount of redundant information between coding units.

[0025] In one embodiment, the processor is configured to send an encoded desktop image to a terminal device, the encoded desktop image including encoding unit indication information, the encoding unit indication information implicitly indicating the position of an encoding unit that includes one of the image units among the plurality of encoding units.

[0026] Encoding unit indication information can implicitly indicate the position of an encoding unit that includes only one image unit in the desktop image, thereby reducing the amount of data in the encoded desktop image.

[0027] Specifically, the coding unit indication information includes the position of the coding unit that includes the plurality of image units among the plurality of coding units. The coding unit indication information can also be used to indicate the size of the coding unit that includes the plurality of image units.

[0028] Furthermore, the coding unit indication information may also include the type of each coding unit, wherein the type of each coding unit is the type of the most numerous image unit among the coding units.

[0029] In one embodiment, the processor is further configured to encrypt the encoding unit indication information, wherein the encoded desktop image includes the encrypted encoding unit indication information.

[0030] The encoding unit indication information is essential for the terminal device to recover the desktop image from the encoded desktop image, and its data volume is relatively small. Encrypting the encoding unit indication information improves the security of transmitting the encoded desktop image while reducing the amount of data that needs to be encrypted, thus increasing the encoding speed. The encrypted data contains a significant amount of redundant data; reducing the amount of data requiring encryption can decrease the consumption of transmission resources.

[0031] In one embodiment, when the color gradient of the image unit is greater than or equal to a preset gradient, the image unit is of a sharp type; when the color gradient of the image unit is less than the preset gradient, the image unit is of a gentle type.

[0032] The encoding unit is determined based on the color gradient of the image unit. This makes the encoding method used when encoding the image unit more consistent with the color gradient of each image unit in the encoding unit, thereby improving the accuracy between the encoded desktop image and the original desktop image.

[0033] Thirdly, a computer program storage medium is provided, characterized in that the computer program storage medium has program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0034] Fourthly, a chip system is provided, characterized in that the chip system includes at least one processor, wherein when program instructions are executed in the at least one processor, the at least one processor causes the at least one processor to perform the method described in the first aspect.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0037] Figure 1 This is a schematic diagram of a cloud service system provided in an embodiment of this disclosure;

[0038] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this disclosure;

[0039] Figure 3 This is a schematic flowchart of another image processing method provided in an embodiment of this disclosure;

[0040] Figure 4 This is a schematic diagram of an encoding unit provided in an embodiment of this disclosure;

[0041] Figure 5 This is a schematic diagram of an encoding unit provided in an embodiment of this disclosure;

[0042] Figure 6 This is a schematic diagram of a desktop image provided in an embodiment of this disclosure;

[0043] Figure 7 This is a schematic diagram illustrating a method for dividing a desktop image according to an embodiment of this disclosure;

[0044] Figure 8 This is a schematic diagram of an atomic block type of a desktop image provided in an embodiment of this disclosure;

[0045] Figure 9 This is a schematic diagram of encoded data provided in an embodiment of this disclosure;

[0046] Figure 10 This is a schematic diagram of encoded data provided in an embodiment of this disclosure;

[0047] Figure 11 This is a schematic structural diagram of an image processing device provided in an embodiment of this disclosure;

[0048] Figure 12This is a schematic structural diagram of another image processing device provided in an embodiment of this disclosure. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0050] To facilitate understanding of the embodiments of this disclosure, let's first take... Figure 1 The cloud service system applicable to the embodiments of this disclosure is described in detail using examples. The cloud service system may also be referred to as a virtual image transmission system or an image transmission system, etc. Figure 1 This is a schematic diagram of a cloud service system.

[0051] Figure 1 The cloud service system shown includes a terminal device 11 and a server 12. The terminal device 11 corresponds to a user, meaning the user can perform corresponding operations on the terminal device 11. The terminal device 11 can be, for example, a computer, mobile phone, tablet computer, laptop computer, set-top box, personal digital assistant (PDA), in-vehicle device, wearable device, or terminal device in the Internet of Things (IoT) or vehicle network. This embodiment does not limit the number of terminal devices 11 included in the image transmission system; for example, there can be one, two, or more. The server 12 can be a single server or a server cluster. The terminal device 11 can also be called a Virtual Graph Transmission Protocol (VGTP) receiver. The server 12 can also be called a virtual desktop server. Data transmission between the terminal device 11 and the server 12 can be based on a local area network (LAN) or a wide area network (WAN).

[0052] Terminal device 11 establishes a remote desktop connection with server 12. Terminal device 11 receives corresponding user operations (such as input via keyboard, mouse or touch screen) and transmits the operation information to server 12.

[0053] Server 12 runs multiple virtual machines (VMs), each corresponding to a terminal device. Server 12 sends operation information sent by terminal device 11 to the corresponding VM, which processes the information to obtain a virtual desktop image. The server captures the virtual desktop image generated by the VM, encodes it, and then transmits it to terminal device 11 via the network. Terminal device 11 decodes and restores the encoded virtual image before displaying it. This virtual image can be a virtual desktop image.

[0054] When encoding a virtual image, the server can divide the image into 16-pixel x 16-pixel macroblocks and perform encoding and decoding on a macroblock-by-macroblock basis. Then, a bitstream is generated based on the encoding result of each macroblock in the virtual image. The server can then send the bitstream to the terminal device.

[0055] Dividing and encoding desktop images into fixed-size macroblocks is applicable to various image scenarios (both natural and computer-generated images) and the algorithm is simple to implement, but the encoded desktop image data volume is still large and the encoding and decoding time is long.

[0056] If larger macroblocks are used to divide and encode the desktop image, the boundaries of the translated image regions will appear in a larger proportion of macroblocks. Therefore, when comparing macroblocks with a reference image, the reference image will include a lower proportion of the same macroblocks, resulting in a larger data volume for the encoded desktop image.

[0057] When encoding the current desktop image, a smaller encoding unit is used. This is because the positions of images in adjacent macroblocks relative to the same images in the reference image change in the same way, but they need to be represented separately. Each macroblock has a lot of additional information, such as information to indicate the macroblock number and information to indicate the position changes of the macroblock's images relative to the reference image, which results in a large amount of data in the encoded desktop image.

[0058] Furthermore, the fixed macroblock size of each frame weakens its security. Although encoded, the encoding process is relatively simple, making it susceptible to interception and decryption. This is particularly problematic in scenarios with high security requirements for image data transmission, such as remote work, where data security cannot be effectively guaranteed.

[0059] To address the aforementioned problems, this disclosure provides an image processing method.

[0060] This disclosure provides an image processing method, such as... Figure 2 As shown.

[0061] This image processing method is applied to devices such as servers, and includes steps 210 to 240.

[0062] 210. Obtain a desktop image, wherein the desktop image comprises multiple image units.

[0063] 220. Determine the type of each image unit based on the image of each image unit.

[0064] The type of an image unit can be determined by whether there is a reference unit image in the reference image that is identical to the image unit.

[0065] The reference image may be an image stored in the device performing steps 210 to 240. The size of the reference image may be the same as the size of the desktop image, for example, it may be the previous frame of the current desktop image.

[0066] If the reference image is the desktop image itself, for an image unit in the desktop image, the type of the image unit can be determined by whether the area outside the image unit in the desktop image includes the image of the image unit. When the area outside the image unit in the desktop image includes the image of the image unit, the type of the image unit is a reference type.

[0067] If the reference image is an image other than the desktop image, the type of the image unit can be determined by whether the reference image includes the image unit's image. When the reference image includes the image unit's image, the type of the image unit is the reference type.

[0068] 230. Based on the type of each image unit, determine multiple encoding units of various sizes in the desktop image, wherein each encoding unit includes at least one image unit, and the maximum number of image units of a certain type in each encoding unit is greater than or equal to a preset number.

[0069] When multiple image units are all of the reference type, the translation vectors of the multiple image units are equal, and the shapes of the multiple image units meet the shape requirements of the first coding unit, the multiple image units constitute the first coding unit.

[0070] In other words, the translation vectors of each image unit in the first coding unit are equal, and the translation vector of each image unit is used to represent the positional relationship between the reference unit image that is the same as the image of the image unit and the image unit. The reference unit image is the same as the image unit in the reference image, and the preset number is the number of image units in the first coding unit.

[0071] The shape of the coding unit can be, for example, a square, i.e., multiple image units are n×n image units, where n is a positive integer. The shape of the coding unit can also be other shapes.

[0072] The translation vector of an image unit is used to represent the positional relationship between that image unit and the same reference unit image in the reference image. Since all image units in the first coding unit are identical, the entire image in the coding unit can be described using only one positional information (e.g., a translation vector) to indicate this positional relationship. This avoids representing multiple image units with the same translation vector using multiple identical translation vectors in the encoded desktop image, thereby reducing redundant information in the encoded desktop image by decreasing the number of coding units.

[0073] The type of an image unit can be determined by analyzing the encoding order of its constituent image units. For example, if image units are encoded from top to bottom, it's possible to determine whether the area above and outside that image unit includes its image, thus identifying the type of that unit. Figure 4 , Figure 5 As shown, it can be determined whether the type of the image unit is a left copy block or an upper copy block.

[0074] When multiple image units are not of the reference type, the number of image units of the smooth or drastic type is greater than or equal to a preset number, and the shape of the multiple image units conforms to the shape requirements of the first coding unit, the multiple image units constitute the first coding unit. It should be understood that the preset number exceeds half of the number of the multiple image units.

[0075] The smooth or abrupt type of an image unit is determined based on its color gradient. When the color gradient is less than a preset value, the image unit is of the smooth type; conversely, when the color gradient is greater than or equal to the preset value, the image unit is of the abrupt type.

[0076] It should be understood that it is also possible to determine whether an image unit is gentle or drastic based solely on its color gradient, without comparing each image unit in the desktop image with a reference unit.

[0077] 240. Encode the desktop image according to the plurality of encoding units.

[0078] After step 240, the encoded desktop image can be sent to the terminal device.

[0079] The encoded desktop image may include encoding unit indication information.

[0080] The coding unit indication information can indicate the position of the coding unit that includes one of the image units among the plurality of coding units, either explicitly or implicitly.

[0081] Encoding unit indication information can implicitly indicate the position of an encoding unit that includes only one image unit in the desktop image, thereby reducing the amount of data in the encoded desktop image.

[0082] Specifically, the coding unit indication information includes the position of the coding unit that includes the plurality of image units among the plurality of coding units. The coding unit indication information can also be used to indicate the size of the coding unit that includes the plurality of image units.

[0083] Furthermore, the coding unit indication information may also include the type of each coding unit, wherein the type of each coding unit is the type of the most numerous image unit among the coding units.

[0084] Furthermore, the encoding unit indication information can be encrypted, and the encoded desktop image includes the encrypted encoding unit indication information.

[0085] The encoding unit indication information is essential for the terminal device to recover the desktop image from the encoded desktop image, and its data volume is relatively small. Encrypting the encoding unit indication information improves the security of transmitting the encoded desktop image while reducing the amount of data that needs to be encrypted, thus increasing the encoding speed. The encrypted data contains a significant amount of redundant data; reducing the amount of data requiring encryption can decrease the consumption of transmission resources.

[0086] The terminal device reconstructs the desktop image based on the received encoded desktop image. It should be understood that if the reference image is an image other than the desktop image, the terminal device stores the reference image.

[0087] The image processing method provided in this disclosure reduces the number of encoding units in a desktop image and reduces redundant information in the encoded desktop image. The type of each image unit is determined based on its image, and the encoding unit of the desktop image is determined based on the type of each image unit. This makes the determination of encoding units more consistent with the different needs of various regions of the desktop image, improving the accuracy between the encoded desktop image and the original desktop image.

[0088] Based on the above Figure 2 The image processing method provided in the corresponding embodiment is disclosed in another embodiment of the present disclosure, which can be applied to a cloud service system.

[0089] Reference Figure 3As shown, the image processing method provided in this embodiment includes the following steps 301 to 304.

[0090] 301. The server determines the type of each atom block in the desktop image.

[0091] The server can divide the desktop image into multiple atomic blocks. Each atomic block has the same size, for example, 16 pixels × 16 pixels.

[0092] The server can classify data according to the size of atomic blocks, such as... Figure 6 The desktop image shown is divided into multiple atomic blocks. For example... Figure 7 As shown, each small square can represent a block of atoms.

[0093] A Cartesian coordinate system can be established, with coordinates (x, y) representing each atom block on the desktop image. In other words, the identifier of an atom block can be coordinates. For example, (0,0) represents the coordinates of the atom block in the first row and first column of the desktop image, (0,1) represents the coordinates of the atom block in the first row and second column at the top left corner of the desktop image, and (1,0) represents the coordinates of the atom block in the second row and first column of the desktop image.

[0094] Alternatively, each atom block on the desktop image can be represented by an index number (or identifier). That is, the identifier for an atom block can be an index number. The width and height of the desktop image are usually fixed; for example, a desktop image can be a 1920×1080 pixel image. Each row of the desktop image contains 120 atom blocks. Therefore, the atom block represented by (7,1) can be labeled as (1×120+7=)127. In other words, the atom block in the 7th row and 1st column can be represented as (7,1) or as number 127.

[0095] If the image of an atomic block is the same as the image of the adjacent atomic block to its left, then the type of the atomic block is determined to include a left copy block.

[0096] like Figure 4 As shown, the images in the four atomic blocks of column j+1 are the same as the images in the four atomic blocks in column j to the left of column j+1. All four atomic blocks in column j+1 are upper copy blocks.

[0097] If an atomic block is not a left copy block, and the image of the atomic block is the same as the image of the adjacent atomic block above it, then the type of the atomic block is determined to include an upper copy block.

[0098] like Figure 5As shown, the images in the four atomic blocks of row i+1 are the same as the images in the four atomic blocks at the corresponding positions in row i above row i+1. All four atomic blocks in row i+1 are upper copy blocks.

[0099] If the image of an atomic block is identical to the image at the same location in the reference image, then the type of the atomic block is determined to include invariant atomic blocks.

[0100] If an atomic block is not an invariant atomic block, and the reference image includes an image of that atomic block, then the type of that atomic block is determined to be a translation atomic block. The change in position of the image of the translation atomic block in the current desktop image and the reference image can be represented by a translation vector. That is, after translation, the image of the translation atomic block is the same as the image of the translated atomic block in the reference image after the translation.

[0101] In some embodiments, the type of an atomic block may simultaneously include one or more of the following: left copy block, top copy block, and invariant atomic block. Alternatively, the type of an atomic block may simultaneously include one or more of the following: left copy block, top copy block, and translation atomic block.

[0102] In other embodiments, it may be preferable to consider whether the type of the atomic block is a left copy block, and if the type of the atomic block is not a left copy block, determine whether the type of the atomic block is an upper copy block. Alternatively, it may be preferable to consider whether the type of the atomic block is an upper copy block, and if the type of the atomic block is not an upper copy block, determine whether the type of the atomic block is a left copy block.

[0103] We can prioritize determining whether the type of the atomic block is a left copy block or a top copy block. If the type of the atomic block is neither a left copy block nor a top copy block, we can then determine whether the type of the atomic block is an invariant atomic block or a translation atomic block. Alternatively, we can also prioritize determining whether the type of the atomic block is an invariant atomic block or a translation atomic block. If the type of the atomic block is neither an invariant atomic block nor a translation atomic block, we can then determine whether the type of the atomic block is a left copy block or a top copy block.

[0104] If an atomic block is not a left copy block, top copy block, invariant atomic block, or translation atomic block, and the color gradient in the atomic block is less than a preset value, then the type of the atomic block is determined to be a smooth block.

[0105] Joint Photographic Experts Group (JPEG) encoding is suitable for encoding images with gentle color changes, i.e., color gradients less than a preset value. In other words, when encoding desktop images, JPEG encoding can be used for sharp blocks.

[0106] If an atomic block is not a left copy block, top copy block, invariant atomic block, or translation atomic block, and the color gradient in the atomic block is greater than or equal to the preset value, then the type of the atomic block is determined to be a violent block.

[0107] Huffman coding is suitable for encoding images with large color gradients. In other words, when encoding desktop images, Huffman coding can be used for sharp blocks.

[0108] A small color gradient in an atomic block indicates that the colors within that block are gentle. A large color gradient in an atomic block indicates that the colors within that block change drastically. Generally, the image corresponding to text areas has a larger color gradient; therefore, gentle blocks can also be called image blocks, and drastic blocks can also be called text blocks.

[0109] 302. The server determines the location of multiple coding units in the desktop image based on the type of each atom block in the desktop image.

[0110] The multiple coding units include at least one of a first-level coding unit, a second-level coding unit, a third-level coding unit, and a fourth-level coding unit. The size of the first-level coding unit is the same as the size of the atomic block; the size of the second-level coding unit is 2 atomic blocks × 2 atomic blocks; the size of the third-level coding unit is 3 atomic blocks × 3 atomic blocks; and the size of the fourth-level coding unit is 4 atomic blocks × 4 atomic blocks.

[0111] like Figure 8 As shown, the types of multiple atomic blocks in a desktop image can include left copy blocks, top copy blocks, smooth blocks, and violent blocks.

[0112] The coding unit containing each atomic block can be determined in each row of the desktop image from top to bottom, in left-to-right order. Each coding unit contains atomic blocks of the same type, or the coding unit includes only smooth blocks and abrupt blocks, and the number of smooth blocks or abrupt blocks is greater than or equal to a preset ratio. It should be understood that this preset ratio is greater than 50%.

[0113] like Figure 8 As shown, the secondary and tertiary coding units in the desktop image are indicated by thicker lines, and the order in which the coding units are determined is indicated by the numbers in the diamond-shaped boxes in the upper left corner of each coding unit in the secondary and tertiary coding units.

[0114] The position of a second-level or third-level coding unit in the desktop image can be indicated by the identifier of the first atomic block at the top left corner of each coding unit, as well as the coding unit indicator information. The coding unit indicator information is used to indicate the size of the coding unit.

[0115] After determining the coding unit to which an atomic block in the desktop image belongs, it is determined whether the coding unit to which the next atomic block in the row to which the atomic block belongs has already been determined. If the coding unit to which the next atomic block belongs has not yet been determined, then the coding unit to which the next atomic block belongs is determined. If the coding unit to which the first atomic block to the right belongs has already been determined, then the coding unit to which the next atomic block belongs is skipped.

[0116] When determining the coding unit to which an atomic block in a desktop image belongs, it is sequentially determined whether the atomic block belongs to the fourth-level coding unit, the third-level coding unit, and the second-level coding unit with the atomic block at the top left corner of the atomic block.

[0117] If the atomic block is a smooth block or a violent block, search for the atomic block type to the right and down. Determine if the 4×4 atomic blocks with the atomic block as the top-left atomic block are all smooth blocks or violent blocks, and if 13 or more atomic blocks have the same type as the atomic block.

[0118] If the judgment result is yes, then the atomic block is determined to belong to the fourth-level coding unit of the 4×4 atomic blocks of the top-left atomic block of the atomic block, and the type of the fourth-level coding unit is the same as the type of the atomic block.

[0119] If the result is negative, then determine whether the 3×3 atomic blocks with the atomic block as the top left corner are all smooth blocks or violent blocks, and whether there are 7 or more atomic blocks of the same type as the atomic block.

[0120] If the determination result is yes, then the atomic block is determined to belong to a 3×3 atomic block third-level coding unit with the atomic block as the top-left first atomic block. The type of the third-level coding unit is the same as the type of the atomic block.

[0121] If the result is negative, then determine whether the 2×2 atomic blocks with the atomic block as the top left corner are all smooth blocks or violent blocks, and whether there are 3 or more atomic blocks of the same type as the atomic block.

[0122] If the determination result is yes, then the atomic block is determined to belong to a second-level coding unit of 2×2 atomic blocks with the atomic block as the top-left first atomic block. The type of the second-level coding unit is the same as the type of the atomic block.

[0123] If the result is negative, then it is determined that the atomic block belongs to the encoding unit, and the type of the encoding unit is the same as the type of the atomic block.

[0124] If the atomic block is any of the following types of atomic blocks: left copy block, top copy block, invariant atomic block, or translation atomic block, that is, the atomic block can be obtained by referring to other atomic blocks, then when determining whether the atomic block belongs to a level 4 coding unit, level 3 coding unit, or level 2 coding unit with the atomic block as the top left atomic block, it is required that the type of each atomic block in the coding unit with the atomic block as the top left atomic block is the same, that is, the type of each atomic block in the coding unit is the type of the atomic block.

[0125] For atomic blocks of various types, such as left copy blocks, top copy blocks, invariant atomic blocks, and translation atomic blocks, the image within the atomic block can be determined through a reference method. For example, for a left copy block, only the type of the atomic block needs to be indicated during encoding. Therefore, the type corresponds to the coding unit in the reference method, and the types of all atomic blocks in the coding unit must be completely identical in order to determine the image of each atomic block in the coding unit based on the type of the coding unit.

[0126] For coding units with smooth or drastic block types, all information of each atomic block image within the coding unit is encoded during the encoding process. The difference between JPEG and Huffman coding lies only in the compression effect. Therefore, coding units with smooth or drastic block types have a certain tolerance for differences in the types of atomic blocks within the coding unit.

[0127] 303. The server encodes the desktop image according to the positions of multiple encoding units in the desktop image.

[0128] In the encoded data, the second-level, third-level, and fourth-level coding units of the desktop image can be recorded in separate data groups. For details on the format of these data groups, please refer to [link to relevant documentation]. Figure 9 .

[0129] Each data group may include level information, at least one type information, and at least one positional information. The level information indicates the size of the coding unit corresponding to the data group. Each positional information corresponds to a type information; for example, each positional information corresponds to the type information preceding it. Each positional information indicates the position of a coding unit. The position of a coding unit can be represented by the identifier of the top-left atomic block within that coding unit. The type information corresponding to each positional information indicates the type of the coding unit indicated by that positional information.

[0130] In each data group, the number of type information N is the same as the number of location information N, which is the number of coding units corresponding to that data group in the desktop image.

[0131] For a desktop image with a resolution of 1920×1080 pixels, the number of atomic blocks including 4×4 pixels is 8038. The number of atomic blocks in the second-level coding unit to the fourth-level coding unit is greater than or equal to 4. Therefore, the number of coding units in a desktop image is less than or equal to 8038.

[0132] The position of the first-level coding unit can no longer be recorded. Once the positions of the second- to fourth-level coding units are determined, the type information of the first-level coding units can be recorded sequentially from left to right within each row, from the first row to the last. The data groups corresponding to the first-level coding units can be found in [reference needed]. Figure 10 The number M of type information for the first-level coding unit is the number of atomic blocks in the desktop image located outside the second- to fourth-level coding units.

[0133] After determining the positions of the Level 2 to Level 4 coding units, the types of atomic blocks not covered by the Level 2 to Level 4 coding units are filled in sequentially from left to right in each row, from the first row to the last row, so that the type of each atomic block in the desktop image can be determined.

[0134] 304. The server encrypts the data group to obtain encrypted data.

[0135] 305. The server sends an encoded desktop image to the terminal device, the encoded desktop image including the encrypted data.

[0136] 306. The terminal device restores the desktop image based on the encoded current desktop image.

[0137] The terminal device decrypts the encrypted data and restores the desktop image based on the decryption result and other information in the encoded current desktop image.

[0138] The image processing method provided in this disclosure reduces the number of encoding units in a desktop image and reduces redundant information in the encoded desktop image. The type of each image unit is determined based on its image, and the encoding unit of the desktop image is determined based on the type of each image unit. This makes the determination of encoding units more consistent with the different needs of various regions of the desktop image, improving the accuracy between the encoded desktop image and the original desktop image.

[0139] Based on the above Figures 2 to 3 The image processing methods described in the corresponding embodiments are described below as device embodiments of this disclosure, which can be used to execute the method embodiments of this disclosure.

[0140] This disclosure provides an image processing device, such as... Figure 11 As shown. The image processing device 500 can be a server. The image processing device 500 includes: a storage module 501 and a processing module 502.

[0141] Storage module 501 is used to store programs.

[0142] When the program is executed in the processing module 502, the processing module 502 is used to execute the image processing method described above.

[0143] Processing module 502 is used for:

[0144] Acquire a desktop image, wherein the desktop image comprises multiple image units;

[0145] The type of each image unit is determined based on the image of each image unit;

[0146] Based on the type of each image unit, multiple coding units of various sizes are determined in the desktop image, wherein each coding unit includes at least one image unit, and the maximum number of image units of a certain type in each coding unit is greater than or equal to a preset number;

[0147] The desktop image is encoded according to the plurality of encoding units.

[0148] Optionally, in the plurality of coding units, the translation vector of each image unit in the plurality of image units of the first coding unit is equal, and the translation vector of each image unit is used to represent the positional relationship between the image unit and the reference unit image that is the same as the image unit. The reference unit image is the same as the image unit in the reference image, and the preset number is the number of image units in the first coding unit.

[0149] Optionally, the processing module 502 is further configured to send an encoded desktop image to the terminal device, the encoded desktop image including encoding unit indication information, the encoding unit indication information implicitly indicating the position of the encoding unit that includes one of the image units among the plurality of encoding units.

[0150] Optionally, the processing module 502 is further configured to encrypt the encoding unit indication information, wherein the encoded desktop image includes the encrypted encoding unit indication information.

[0151] Optionally, when the color gradient of the image unit is greater than or equal to a preset gradient, the image unit is of a sharp type; when the color gradient of the image unit is less than the preset gradient, the image unit is of a gentle type.

[0152] The image processing device provided in this disclosure simplifies user operations during the user login process and improves user experience.

[0153] Based on the above Figures 2 to 3The image processing method described in the corresponding embodiments also provides an image processing apparatus in this disclosure, such as... Figure 12 As shown.

[0154] The image processing apparatus 600 includes a memory 601 and a processor 602. The image processing apparatus 600 may be a server.

[0155] The memory 601 is used to store program instructions.

[0156] When the program is executed in processor 602, processor 602 is used to execute the image processing method described above.

[0157] Processor 602 is used for:

[0158] Acquire a desktop image, wherein the desktop image comprises multiple image units;

[0159] The type of each image unit is determined based on the image of each image unit;

[0160] Based on the type of each image unit, multiple coding units of various sizes are determined in the desktop image, wherein each coding unit includes at least one image unit, and the maximum number of image units of a certain type in each coding unit is greater than or equal to a preset number;

[0161] The desktop image is encoded according to the plurality of encoding units.

[0162] Optionally, in the plurality of coding units, the translation vector of each image unit in the plurality of image units of the first coding unit is equal, and the translation vector of each image unit is used to represent the positional relationship between the image unit and the reference unit image that is the same as the image unit. The reference unit image is the same as the image unit in the reference image, and the preset number is the number of image units in the first coding unit.

[0163] Optionally, the processor 602 is further configured to send an encoded desktop image to a terminal device, the encoded desktop image including encoding unit indication information, the encoding unit indication information implicitly indicating the position of an encoding unit that includes one of the image units among the plurality of encoding units.

[0164] Optionally, the processor 602 is further configured to encrypt the encoding unit indication information, wherein the encoded desktop image includes the encrypted encoding unit indication information.

[0165] Optionally, when the color gradient of the image unit is greater than or equal to a preset gradient, the image unit is of a sharp type; when the color gradient of the image unit is less than the preset gradient, the image unit is of a gentle type.

[0166] Based on the above Figures 2 to 3 In addition to the image processing method described in the corresponding embodiments, this disclosure also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage device. This storage medium stores computer instructions for executing the above-described methods. Figure 2 and Figure 3 The image processing methods applied to terminal devices or servers described in the corresponding embodiments will not be repeated here.

[0167] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0168] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire a desktop image, wherein the desktop image comprises multiple image units; The type of each image unit is determined based on the image of each image unit; Based on the type of each image unit, multiple coding units of various sizes are determined in the desktop image, wherein each coding unit includes at least one image unit, and the maximum number of image units of a certain type in each coding unit is greater than or equal to a preset number; The desktop image is encoded according to the plurality of encoding units; The method further includes: The encoded desktop image is sent to the terminal device. The encoded desktop image includes encoding unit indication information, which implicitly indicates the position of the encoding unit that includes one of the image units among the plurality of encoding units. The encoding unit indication information also includes the type of each encoding unit, and the type of each encoding unit is the type of the most numerous image unit among the encoding units. In the plurality of coding units, the translation vector of each image unit in the plurality of image units of the first coding unit is equal. The translation vector of each image unit is used to represent the positional relationship between the image unit and the reference unit image that is the same as the image unit. The reference unit image is the same as the image unit in the reference image. The preset number is the number of image units in the first coding unit.

2. The method according to claim 1, characterized in that, The method further includes: The encoding unit indication information is encrypted, and the encoded desktop image includes the encrypted encoding unit indication information.

3. The method according to any one of claims 1-2, characterized in that, When the color gradient of the image unit is greater than or equal to a preset gradient, the type of the image unit is severe; When the color gradient of the image unit is less than the preset gradient, the type of the image unit is flat.

4. An image processing apparatus, characterized in that, Including memory and processor; The memory has a stored program; When the program is executed in the processor, the processor is used to: Acquire a desktop image, wherein the desktop image comprises multiple image units; The type of each image unit is determined based on the image of each image unit; Based on the type of each image unit, multiple coding units of various sizes are determined in the desktop image, wherein each coding unit includes at least one image unit, and the maximum number of image units of a certain type in each coding unit is greater than or equal to a preset number; The desktop image is encoded according to the plurality of encoding units; The processor is configured to send an encoded desktop image to a terminal device. The encoded desktop image includes encoding unit indication information, which implicitly indicates the position of an encoding unit that includes one of the image units among the plurality of encoding units. The encoding unit indication information also includes the type of each encoding unit, where the type of each encoding unit is the type of the most numerous image unit among the encoding units. In the plurality of coding units, the translation vector of each image unit in the plurality of image units of the first coding unit is equal. The translation vector of each image unit is used to represent the positional relationship between the image unit and the reference unit image that is the same as the image unit. The reference unit image is the same as the image unit in the reference image. The preset number is the number of image units in the first coding unit.

5. The apparatus according to claim 4, characterized in that, The processor is further configured to encrypt the encoding unit indication information, wherein the encoded desktop image includes the encrypted encoding unit indication information.

6. The apparatus according to any one of claims 4-5, characterized in that, When the color gradient of the image unit is greater than or equal to a preset gradient, the type of the image unit is severe; When the color gradient of the image unit is less than the preset gradient, the type of the image unit is flat.

Images