Image encoding, decoding method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide an image encoding and decoding method, device, electronic device and storage medium, wherein the image encoding method includes: determining the display level of the display layer in the screen image; determining the top display layer in the screen image according to the display level, and determining the first display area of ​​interest to the user in the screen image at least based on the top display layer; setting the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, the second display area being the display area outside the first display area in the screen image; encoding the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image. The embodiments of the present application can improve the image encoding efficiency while taking into account both image quality and image compression rate.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and specifically to an image encoding and decoding method, device, electronic device and storage medium. Background Art

[0002] Image coding is a technology that uses image compression to represent image content with a smaller amount of data. It is widely used in many fields such as image editing and image transmission. Image quality and image compression rate are two important indicators of image coding technology. The higher the image quality, the clearer the encoded image, and the higher the image compression rate, the smaller the amount of data of the encoded image. Therefore, image coding technology mainly reduces the amount of image data by increasing the image compression rate as much as possible while meeting the image quality. In this context, how to balance image quality and image compression rate and efficiently complete image coding has become a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an image encoding and decoding method, device, electronic device and storage medium to improve the image encoding efficiency while taking into account both image quality and image compression rate.

[0004] To achieve the above objectives, the present application provides the following technical solutions:

[0005] An image encoding method, applied to an electronic device, comprising:

[0006] Determine the display level of the display layer in the screen image;

[0007] Determine a top display layer in the screen image according to the display hierarchy, and determine a first display area of ​​interest to a user in the screen image at least based on the top display layer;

[0008] Setting the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, where the second display area is a display area outside the first display area in the screen image;

[0009] The screen image is encoded according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image.

[0010] The present application also provides an image decoding method, which is applied to an electronic device, and the method includes:

[0011] Acquire an encoded image corresponding to a screen image, wherein the compressed image quality of a first display area of ​​interest to a user in the encoded image is higher than the compressed image quality of a second display area, the second display area being a display area outside the first display area, and the first display area at least including a top display layer in the screen image;

[0012] The encoded image is decoded.

[0013] The present application also provides an image encoding device, including:

[0014] A display level determination module, used to determine the display level of the display layer in the screen image;

[0015] an interest region determining module, configured to determine a top display layer in the screen image according to the display hierarchy, and to determine a first display region in the screen image that is of interest to the user based at least on the top display layer;

[0016] A compression setting module, used to set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, where the second display area is a display area outside the first display area in the screen image;

[0017] The encoding module is used to encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image.

[0018] The embodiment of the present application further provides an image encoding device, including: an encoder; the encoder is used to:

[0019] Determine a display level of a window in a screen image; determine a top-level window in the screen image according to the display level, and determine a first display area of ​​interest to a user in the screen image at least based on the top-level window; set a compressed image quality of the first display area to be higher than a compressed image quality of a second display area in the screen image, the second display area being a display area outside the first display area in the screen image; encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image.

[0020] The present application also provides an image decoding device, including:

[0021] A coded image acquisition module, used to acquire a coded image corresponding to a screen image, wherein the compressed image quality of a first display area of ​​interest to a user in the coded image is higher than the compressed image quality of a second display area, the second display area being a display area outside the first display area, and the first display area at least including a top display layer in the screen image;

[0022] A decoding module is used to decode the encoded image.

[0023] An embodiment of the present application also provides an electronic device, comprising at least one memory and at least one processor; the memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to execute the image encoding method as described above, or the image decoding method as described above.

[0024] An embodiment of the present application also provides a storage medium, which stores one or more computer-executable instructions, and when the one or more computer-executable instructions are executed, the image encoding method as described above, or the image decoding method as described above is implemented.

[0025] The image encoding method provided in the embodiment of the present application can encode a screen image. For the display layer in the screen image, the embodiment of the present application can obtain the display level of the display layer, and determine the top display layer in the screen image according to the display level. Since the user is likely to be interested in the top display layer in the screen image, the embodiment of the present application can determine the first display area in the screen image that the user is interested in based on at least the top display layer; thus, for the first display area in the screen image, and the second display area outside the first display area, the embodiment of the present application can set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area, so that the first display area has a higher image quality, while the image bit rate of the second display area is reduced (that is, the image data volume of the second display area is reduced); further, the embodiment of the present application encodes the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain a coded image, which can reduce the overall data volume of the encoded image and improve the image compression rate while ensuring the image quality of the first display area in the screen image.

[0026] At the same time, since the first display area of ​​interest to the user in the screen image is determined based on at least the top display layer, that is, the embodiment of the present application can determine the first display area of ​​interest to the user based on the display level of the display layer in the screen image, without the need to use ROI recognition technology to identify the area of ​​interest to the user in the screen image, thereby greatly improving the efficiency of determining the part of interest to the user in the screen image, and realizing image encoding in an efficient manner. It can be seen that the embodiment of the present application can improve the image encoding efficiency while taking into account both image quality and image compression efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0028] Figure 1 A flowchart of an image encoding method provided in an embodiment of the present application;

[0029] Figure 2 This is an example diagram showing the layers;

[0030] Figure 3 A flowchart for determining a first display area provided in an embodiment of the present application;

[0031] Figure 4 is another example diagram of the display layer;

[0032] Figure 5 is another example diagram of the display layer;

[0033] Figure 6 is another example diagram of the display layer;

[0034] Figure 7 A component architecture diagram of an electronic device provided in an embodiment of the present application;

[0035] Figure 8 A flowchart of an image decoding method provided in an embodiment of the present application;

[0036] Fig. 9 Example diagram for desktop sharing service;

[0037] Fig.10 A block diagram of an image encoding device provided in an embodiment of the present application;

[0038] Fig.11 Another block diagram of the image encoding device provided by the embodiment of the present application;

[0039] Fig.12 Another block diagram of an image encoding device provided by an embodiment of the present application;

[0040] Fig.13 Another block diagram of an image encoding device provided in an embodiment of the present application;

[0041] Fig.14 A block diagram of an electronic device provided in an embodiment of the present application;

[0042] Fig.15 A block diagram of an image decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] A current mainstream image coding scheme is to identify the ROI region of interest to the user and the non-ROI region of no interest to the user from the image based on the ROI (Region Of Interest) recognition technology, so as to set a higher compression image quality for the ROI region and a lower compression image quality for the non-ROI region, and then compress the image to complete the image coding; it should be noted that the higher the compression image quality, the higher the quality of the compressed image and the larger the corresponding image data volume, and the lower the compression image quality, the lower the quality of the compressed image and the smaller the corresponding image data volume; the above-mentioned image coding scheme using the ROI recognition technology can make the image quality of the ROI region higher and the image quality of the non-ROI region lower, that is, under the premise of meeting the image quality requirements of the ROI region, reduce the image bit rate of the non-ROI region, so that the overall image data volume is reduced, so as to achieve the effect of improving the image compression rate; it should be noted that the ROI recognition technology mainly relies on image characteristics, and uses machine learning technology or image recognition technology to identify the detailed image content and non-detailed image content in the image, and regards the area where the detailed image content is located as the ROI area, and the area where the non-detailed image content is located as the non-ROI area;

[0045] Although the above-mentioned image coding scheme using ROI recognition technology can ensure the image quality of the ROI area of ​​interest to the user on the basis of identifying the ROI area and the non-ROI area of ​​the image, and improve the image compression rate by reducing the image bit rate of the non-ROI area of ​​no interest to the user, it still has certain limitations, specifically:

[0046] ROI recognition technology needs to be implemented based on machine learning technology or image recognition technology, and machine learning technology or image recognition technology itself has a relatively complex processing logic. Therefore, when encoding an image with specific image content, applying ROI recognition technology to implement image encoding may not be a good choice; for example, when encoding a screen image, there is a display layer (display layer such as a window, etc.) in the screen image, especially when there are multiple display layers in the screen image, because the display layer will block part of the image content, and machine learning technology or image recognition technology itself has a relatively complex processing logic, so using machine learning technology or image recognition technology to identify the detailed image content in the screen image as the ROI area and the non-detailed image content as the non-ROI area may consume higher system resources, which makes the image encoding scheme using ROI recognition technology unable to efficiently complete image encoding when the screen image has a display layer, that is, when the screen image has a display layer, the image encoding efficiency of the image encoding scheme using ROI recognition technology is limited;

[0047] Based on this, when encoding a screen image and there is a display layer in the screen image, the inventor of the embodiment of the present application proposes a new image encoding scheme to improve the image encoding efficiency while taking into account both image quality and image compression efficiency.

[0048] As an optional implementation, Figure 1 An optional process of the image encoding method provided by the embodiment of the present application is shown. The image encoding method can be executed by an electronic device. The electronic device can be an electronic device that provides a screen image. A user can implement screen operations based on the screen image provided by the electronic device. The electronic device can encode the screen image to form a coded image by implementing the solution provided by the embodiment of the present application. The coded image can be stored locally in the electronic device or transmitted to other devices.

[0049] In a possible application scenario, the electronic device may be a terminal device, such as a terminal device that may encode a local screen image displayed locally to form a coded image (such as a video coded image in video form, etc.), which may be stored locally on the terminal device or shared with other devices; in a possible application example, taking a screen sharing service as an example, the terminal device may encode a local screen image displayed locally using a scheme provided in an embodiment of the present application to form a video coded image, which may be shared with other terminal devices via a network or a cloud service that provides a screen sharing service, etc.; a specific form of a screen sharing service may be a remote desktop service, a desktop sharing service, etc.;

[0050] In other possible application scenarios, the electronic device may be a cloud service device. For example, when the cloud service device provides a cloud screen service for a terminal device, the cloud service device may use the solution provided in the embodiment of the present application to encode the cloud screen image to form a video encoding image, which may be transmitted to the terminal device, so that the terminal device can implement cloud screen operation after decoding and displaying the video image; a specific form of the cloud screen service may be a cloud desktop service, etc.;

[0051] Optional, see Figure 1 As shown, the image encoding method provided in the embodiment of the present application may include:

[0052] Step S100: Determine the display level of the display layer in the screen image.

[0053] Optionally, the screen image may be a local screen image corresponding to the local screen of the terminal device, and accordingly, the terminal device may execute the image encoding method provided in the embodiment of the present application to encode the local screen image displayed on the local screen of the terminal device; in other optional implementations, the screen image may be a cloud screen image corresponding to the cloud screen service of the cloud service device, and accordingly, the cloud service device may execute the image encoding method provided in the embodiment of the present application to encode the cloud screen image.

[0054] In the embodiment of the present application, no matter the screen image is a local screen of a terminal device or a cloud screen image corresponding to a cloud screen service, the screen image may have a display layer, the number of display layers may be one or more, and the display layer may be understood as a component in the screen image used to display image content, such as a window, icon, etc. used to display image content in the screen image. It is understandable that the image content of the screen image may include image content outside the display layer in addition to the image content of each display layer, such as the image content of the screen background, etc.; for example, taking the screen image as a desktop image and the display layer as a window as an example, the image content of the desktop image may include image content of the desktop background in addition to the image content of each window.

[0055] In a possible implementation, the display layer system of the electronic device (whether it is a terminal device or a cloud service device) can be used to manage the display layers in the screen. The display layer system can obtain information such as the display level, position and resolution of each display layer in the screen, so that the embodiment of the present application can at least determine the display level of the display layer in the screen image through the display layer system. It can be understood that if there is only one display layer in the screen image, the display level of the display layer is single, and the one display layer is located at the top layer. If there are multiple display layers in the screen image, the multiple display layers have multiple display levels, and the display layer at the upper display level is superimposed on the display layer at the next display level;

[0056] In a more specific example, taking the screen image as a desktop image and the display layer as a window as an example, the window system of the electronic device can be used to manage the windows in the screen, so that the display level, position, resolution and other information of each window can be obtained through the window system. It can be understood that if the desktop only displays one window, the display level of the window is single, and the one window is at the top layer, and if the desktop displays multiple windows, the multiple windows have multiple display levels, and the window at the upper display level is superimposed on the window at the next display level.

[0057] Step S110: determine a top display layer in the screen image according to the display hierarchy, and determine a first display area in the screen image that is of interest to the user based at least on the top display layer.

[0058] The inventors of the embodiments of the present application have found through research that when a user operates the screen, the user is likely to place the display layer of interest (such as a window) on the top layer so that the user can perform data editing, content viewing and other operations through the top display layer. Therefore, after determining the display level of the display layer in the screen image, the embodiments of the present application can determine the top display layer at the top of the screen image according to the display level (for ease of explanation, the display layer at the top of the screen image can be referred to as the top display layer), so that the display area corresponding to the top display layer should at least serve as the first display area of ​​interest to the user in the screen image (for ease of explanation, the display area of ​​interest to the user in the screen image can be referred to as the first display area). It can be understood that the first display area of ​​interest to the user in the screen image needs to ensure image quality during the encoding process.

[0059] In an optional implementation, the embodiment of the present application may only use the display area corresponding to the top-level display layer as the first display area of ​​interest to the user in the screen image. In this case, no matter whether there is one or more display layers in the screen image, that is, no matter whether there is one or more display layers in the screen image, after determining the top-level display layer, the embodiment of the present application only uses the display area corresponding to the top-level display layer as the first display area.

[0060] In another optional implementation, the first display area of ​​interest to the user in the screen image may include other display areas in addition to the display area corresponding to the top display layer; for example, in the case where there are multiple display layers in the screen image, the other display areas of interest to the user may involve a display layer different from the top display layer, such as a display layer next to the top display layer, or one or more display layers specified by the user in addition to the top display layer; for ease of explanation, in the embodiment of the present application, the display layer (which may be one or more) that the user is interested in in addition to the top display layer in the screen image is referred to as a target display layer, and the target display layer may be a display layer next to the top display layer, or may be one or more display layers specified by the user in addition to the top display layer;

[0061] Optionally, the next display layer may be considered to be a display layer one level lower than the top display layer; for example, Figure 2 As shown, the screen displays display layers 1, 2, and 3 (in this example, the display layer can be understood as, for example, a window, etc.), wherein, based on the display levels of display layers 1, 2, and 3, display layer 1 is the top display layer, the display level of display layer 2 is one level lower than display layer 1, and is the next display layer of display layer 1, and the display level of display layer 3 is one level lower than display layer 2, and the display level of display layer 3 is one level lower than display layer 2. Figure 2 For example, the first display area of ​​interest to the user in the screen image may include, in addition to the display area corresponding to the top display layer 1, the display area corresponding to the next display layer 2 of the top display layer; it should be explained that when the user operates the screen (such as performing data editing based on the screen) or views the screen, the user may need to combine operations on multiple display layers (such as multiple windows) to complete the screen operation or view the screen. At this time, the user is likely to implement the screen operation or view the screen based on the top display layer and the next display layer of the top display layer. That is, at this time, in addition to being interested in the top display layer, the user also pays attention to the next display layer of the top display layer. Therefore, the display area corresponding to the next display layer can also be regarded as the first display area of ​​interest to the user, so as to ensure the image quality during the encoding process of the screen image;

[0062] In other optional implementations, the target display layer may also be specified by a specific operation of the user or a set shortcut operation. That is, the embodiment of the present application also allows the user to specify the target display layer of interest when the top-level display layer is set by default as the display layer of interest to the user. For example, when the user opens a certain display layer, the user can specify the opened display layer as the display layer of interest through a specific operation (such as a specific operation of adding an interest mark, etc.) or a set shortcut operation.

[0063] Step S120: Set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, where the second display area is a display area outside the first display area in the screen image.

[0064] After determining the first display area in the screen image, since the user is likely to be interested in the image content of the first display area, a higher compression image quality needs to be set for the first display area. For the second display area outside the first display area in the screen image, since the user is likely not to be interested in the image content of the second display area, a lower compression image quality can be set for the second display area, so as to reduce the image bit rate of the second display area and reduce the amount of image data in the second display area during the image encoding process.

[0065] As an optional implementation, in a lossy coding algorithm of an image (such as a video image), the quantization coefficient is related to the coding quality and coding bit rate of the image, and is one of the factors that determine the image quality. In image coding, the quantization coefficient is the serial number of the quantization step. The quantization coefficient is positively correlated with the quantization loss of the image and negatively correlated with the image quality, that is, the smaller the value of the quantization coefficient, the finer the coefficient quantized by the quantization step, the smaller the quantization loss of the image, and the higher the image quality; conversely, the larger the value of the quantization coefficient, the coarser the coefficient quantized by the quantization step, the greater the quantization loss of the image, and the lower the image quality, that is, the quantization coefficient is negatively correlated with the image quality;

[0066] Based on this, the embodiment of the present application can set quantization coefficients for the first display area and the second display area to achieve setting compressed image quality for the first display area and the second display area, that is, the embodiment of the present application can set the quantization coefficient of the first display area to be lower than the quantization coefficient of the second display area, so that for the first display area in the screen image that the user is interested in, and the second display area outside the first display area that the user is most likely not interested in, the embodiment of the present application sets the quantization coefficient of the first display area to be lower than the quantization coefficient of the second display area, so that the image quality of the image content of the first display area is higher than the image quality of the image content of the second display area. On the premise of ensuring the image quality of the first display area, the image data amount of the second display area is reduced, so that the overall image data amount after the screen image is encoded is reduced, thereby achieving the effect of improving the image compression rate.

[0067] Of course, setting the compressed image quality by setting the quantization coefficient is only an optional means, and the embodiment of the present application can also support other parameters that can reflect the compressed image quality.

[0068] In an optional implementation, if the screen image has only one display layer, and the display layer is the top display layer, then the display area corresponding to the top display layer can be directly used as the first display area, and the second display area in the screen image other than the first display area can be regarded as the background area of ​​the screen image; accordingly, in this case, the embodiment of the present application can set the quantization coefficient of the top display layer to be lower than the quantization coefficient of the background area of ​​the screen image;

[0069] In another optional implementation, if the screen image has multiple display layers, the multiple display layers may include a top display layer, a target display layer that the user is interested in except the top display layer, and a non-target display layer that the user is not interested in; for example, in combination with Figure 2 As shown in the example, the screen image has display layers 1, 2 and 3, display layer 1 is the top display layer, display layer 2 is the target display layer of interest to the user except the top display layer, and display layer 3 is the non-target display layer. The display area corresponding to display layer 1 and display layer 2 is used as the first display area, and the display area corresponding to display layer 3 and the background area of ​​the screen image can be used as the second display area. In this case, the embodiment of the present application can set the quantization coefficients of the top display layer and the target display layer to be the same, and lower than the quantization coefficients of the non-target display layer and the background area in the screen image.

[0070] It should be noted that the background area of ​​the screen image can be considered as the display area of ​​the screen image except the display layer, for example Figure 2 In the example, the display area except display layers 1, 2 and 3 can be regarded as the background area of ​​the screen image.

[0071] Step S130: Encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain a coded image.

[0072] In an optional implementation, the embodiment of the present application may encode the screen image based on the quantization coefficient of the first display area and the quantization coefficient of the second display area to obtain an encoded image corresponding to the screen image, thereby completing the image encoding of the screen image.

[0073] In a further optional implementation, the encoded image may be stored by the electronic device that performs the image encoding, or may be transmitted by the electronic device to another device so that the encoded image can be decoded on the other device and the decoded image can be displayed;

[0074] In a possible application scenario, taking the example of a terminal device using a screen sharing service to share a screen with other terminal devices, after the terminal device uses the solution provided in the embodiment of the present application to encode a local screen image corresponding to the local screen, the terminal device can share the encoded image with other terminal devices participating in the screen sharing service, thereby realizing screen sharing between multiple terminal devices with a lower amount of image data;

[0075] In other possible application scenarios, taking the example of a cloud service device providing cloud screen services to a terminal device, the cloud service device can use the solution provided in the embodiment of the present application to encode the cloud screen image, and transmit the encoded image to the terminal device using the cloud screen service, thereby providing the cloud screen service to the terminal device with a lower amount of image data.

[0076] It should be noted that, in addition to the effect of improving the image compression rate while ensuring the image quality of the first display area, the embodiment of the present application also has the advantage of higher image coding efficiency. Specifically, the embodiment of the present application determines the first display area of ​​interest to the user in the screen image by determining the display level of the display layer in the screen image, thereby at least based on the top display layer in the screen image. Therefore, when there is a display layer in the screen image, the embodiment of the present application can determine the part of the screen image that the user is interested in based at least on the display level of the display layer in the screen image, without the need to use ROI recognition technology to identify the area of ​​interest to the user in the screen image, thereby reducing the system resources required to determine the part of the screen image that the user is interested in, and can achieve efficient determination of the part of the screen image that the user is interested in, thereby improving the image coding efficiency, and can improve the image coding efficiency while taking into account both image quality and image compression rate.

[0077] The image encoding method provided in the embodiment of the present application can encode a screen image. For the display layer in the screen image, the embodiment of the present application can obtain the display level of the display layer, and determine the top display layer in the screen image according to the display level. Since the user is likely to be interested in the top display layer in the screen image, the embodiment of the present application can determine the first display area in the screen image that the user is interested in based on at least the top display layer; thus, for the first display area in the screen image, and the second display area outside the first display area, the embodiment of the present application can set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area, so that the first display area has a higher image quality, while the image bit rate of the second display area is reduced (that is, the image data volume of the second display area is reduced); further, the embodiment of the present application encodes the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain a coded image, which can reduce the overall data volume of the encoded image and improve the image compression rate while ensuring the image quality of the first display area in the screen image.

[0078] At the same time, since the first display area of ​​interest to the user in the screen image is determined based on at least the top display layer, that is, the embodiment of the present application can determine the first display area of ​​interest to the user based on the display level of the display layer in the screen image, without the need to use ROI recognition technology to identify the area of ​​interest to the user in the screen image, thereby greatly improving the efficiency of determining the part of interest to the user in the screen image, and realizing image encoding in an efficient manner. It can be seen that the embodiment of the present application can improve the image encoding efficiency while taking into account both image quality and image compression efficiency.

[0079] In view of the situation where there are multiple display layers in the screen image, a possible implementation scheme for determining the first display area of ​​interest to the user in the screen image according to an embodiment of the present application is introduced below.

[0080] As an optional implementation, Figure 3 An optional process for determining the first display area provided in an embodiment of the present application is shown. Figure 3 As shown, the process may include:

[0081] Step S200, determine whether there is a target display layer that the user is interested in in the screen image in addition to the top display layer, if so, execute step S210, if not, execute step S220.

[0082] Step S210: Determine the display area corresponding to the top display layer and the target display layer as the first display area of ​​interest to the user in the screen image.

[0083] Step S220: determine the display area corresponding to the top display layer as the first display area of ​​interest to the user in the screen image.

[0084] In addition to using the display area corresponding to the top display layer as the first display area of ​​interest to the user, the embodiment of the present application may further analyze whether there is a target display layer of interest to the user in the display layer of the screen image. Thus, when there is a target display layer of interest to the user in the screen image, the embodiment of the present application may also use the display area corresponding to the target display layer as the first display area, thereby ensuring the image quality of the top display layer and the target display layer during the image encoding process. If there is no target display layer of interest to the user in the screen image except the top display layer, the embodiment of the present application may only use the display area corresponding to the top display layer as the first display area.

[0085] In a possible implementation, the target display layer may be a display layer next to the top display layer. The embodiment of the present application may analyze whether there is a target display layer in which the user is interested in the screen image by analyzing whether the user is interested in the display layer next to the top display layer. Accordingly, if it is analyzed that the user is interested in the next display layer, the next display layer is used as the target display layer. The embodiment of the present application may determine the display area corresponding to the top display layer and the next display layer as the first display area.

[0086] In other possible implementations, the user may specify the target display layer. The embodiment of the present application may determine whether there is a target display layer of interest specified by the user in the screen image. If it is determined that there is a target display layer of interest specified by the user in the screen image, the embodiment of the present application may determine the display area corresponding to the top display layer and the specified target display layer as the first display area.

[0087] In one example, the target display layer of interest specified by the user is, for example, a target display layer with an interest tag added by the user. The embodiment of the present application can determine whether there is a target display layer with an interest tag added in the screen image in addition to the top display layer. Thus, when determining whether there is a target display layer with an interest tag added in the screen image, the embodiment of the present application can determine the display area corresponding to the top display layer and the target display layer with the interest tag added as the first display area.

[0088] More specifically, the target display layer is used as the next display layer of the top display layer for explanation. Judging whether the user is interested in the next display layer mainly involves analyzing whether the user will pay attention to the image content of the next display layer. Specifically, different analysis strategies can be provided based on different scenarios in which the user operates the screen (editing data and / or viewing content) in combination with the top display layer and the next display layer. The following only further explains several possible situations in which the user is interested in the next display layer.

[0089] Case 1: In the embodiment of the present application, it is possible to determine whether the user is interested in the next display layer at least based on the display content of the next display layer. For example, when the display content of the next display layer is the content that the user is interested in, it can be determined that the user is interested in the next display layer, and the next display layer can be used as the target display layer. In a possible implementation, the display content of the next display layer is the content that the user is interested in. For example, the type of the display content of the next display layer is the type that the user is interested in (such as by analyzing the user preference, to determine whether the type of the display content of the next display layer is preferred by the user, etc.), and for example, the next display layer displays dynamic content (such as video, animation and other dynamic content). Since the content of the next display layer has a large change in the time domain when displaying dynamic content, it is possible that the user sets the display layer displaying dynamic content as the next display layer of the top display layer intentionally, that is, the user may intentionally place the display layer displaying dynamic content at the next display level of the top display layer, so that the user is paying attention to the dynamic content displayed by the next display layer when paying attention to the top display layer. Possible scenarios of this situation are, for example, the top window is a chat window, and the next window of the top window is a movie playback window, that is, the user chats while watching a movie, etc.;

[0090] For situation one, the embodiment of the present application may determine whether the user is interested in the next display layer at least according to the display content of the next display layer; for example, the embodiment of the present application may at least determine whether the next display layer displays dynamic content, so as to determine that the user is interested in the next display layer when at least determining that the next display layer displays dynamic content; in a possible implementation, the embodiment of the present application may determine whether the next display layer displays dynamic content through a display layer activity detection (such as window activity detection) function, so as to determine that the user is interested in the next display layer at least when the judgment result of the display layer activity detection function is yes, the display layer activity detection function can be understood as a function of detecting whether the content of the display layer changes over time;

[0091] In further optional implementations, there may be some more detailed situations in situation 1: when determining that the next display layer displays dynamic content, the embodiment of the present application can directly determine that the user is interested in the next display layer, that is, as long as the next display layer displays dynamic content, regardless of other display attributes of the next display layer, the embodiment of the present application can directly determine that the user is interested in the next display layer; as another optional implementation of situation 1, when determining that the next display layer displays dynamic content, the embodiment of the present application also needs to combine the proportion of the area of ​​the next display layer that is not blocked to further determine whether the user is interested in the next display layer, which is specifically described as follows:

[0092] Although the next display layer displays dynamic content, if the area of ​​the next display layer that is blocked is too large, and since the blocked area of ​​the display layer can be deliberately set by the user, the user may not be interested in the dynamic content displayed by the next display layer at this time; based on this, the embodiment of the present application can further determine the unblocked area ratio of the next display layer, and judge whether the unblocked area ratio of the next display layer is not less than the set first ratio, so that when it is determined that the unblocked area ratio of the next display layer is not less than the first ratio, it is determined that the user is interested in the next display layer, that is, the unblocked area ratio of the next display layer is not less than the set first ratio, indicating that most of the area of ​​the next display layer is not blocked, and the user is very likely to pay attention to both the top display layer and the next display layer at this time;

[0093] Example, combined Figure 4 As shown, window 1 is the top-level window, window 2 is the next window, and window 2 displays dynamic content. Since most of the area ratio of window 2 is not blocked, when the user pays attention to window 1, it is highly likely that the user will also pay attention to window 2. In order to reduce the misjudgment of the user's area of ​​interest, the embodiment of the present application can expand the area of ​​interest to the user on the basis of window 1 (the top-level display layer), and it is considered that the user is also interested in window 2. Therefore, the embodiment of the present application can use the display area corresponding to window 1 and window 2 as the first display area; it should be noted that the embodiment of the present application can determine the position information of the display layer in the screen image while determining the display level of the display layer in the screen image. The position information of the display layer can include the position of the boundary points of the display layer, so that the display area of ​​the display layer can be the area surrounded by the position of the boundary points of the display layer;

[0094] It should be pointed out that the proportion of the unobstructed area of ​​the next display layer can be understood as the unobstructed area of ​​the next display layer divided by the area of ​​the next display layer, and the first proportion mentioned above can be a pre-set proportion of the area in which most of the content of the display layer is considered to be unobstructed, such as 60%, etc. Of course, the value here is only an example, and the specific value of the first ratio can be set according to the actual situation.

[0095] Case 2: In the embodiment of the present application, whether the user is interested in the next display layer can be determined based on the area of ​​the top display layer and the area relationship between the top display layer and the next display layer; it is understandable that if the area of ​​the top display layer is small, and the area of ​​the next display layer is larger than the top display layer, from the user's sensory perspective, since the area of ​​the top display layer is small, the user generally will not focus on the top display layer when scaling the area of ​​the top display layer. At this time, the user is very likely to focus on the next display layer whose area is larger than the top display layer. Therefore, the next display layer can be regarded as the target display layer that the user is interested in. This situation may be in the following scenarios: the top window is a music lyrics window, and the next window of the top window is a text window, that is, the user listens to music while viewing text, etc.

[0096] For the second situation, the embodiment of the present application can determine whether the user is interested in the next display layer at least based on the area of ​​the top display layer and the area relationship between the top display layer and the next display layer; for example, when the area of ​​the top display layer is small and the area of ​​the next display layer is larger than the top display layer, the next display layer is considered to be the target display layer that the user is interested in; in a possible implementation, the embodiment of the present application can determine the area of ​​the display layer and the area relationship between the display layers through a display layer validity judgment (such as a window validity judgment) function, and the display layer validity judgment function can be understood as a function of detecting display properties such as the position and area of ​​the display layer;

[0097] In a more specific optional implementation, the embodiment of the present application can determine the area ratio of the top display layer occupying the screen image, and the area ratio of the next display layer to the top display layer, for example, by setting a second ratio with a smaller area ratio and a first multiple with a higher area ratio, to determine whether the area ratio of the top display layer occupying the screen image is not greater than the set second ratio, and whether the area of ​​the next display layer is at least the first multiple of the area of ​​the top display layer, so that when it is determined that the area ratio of the top display layer occupying the screen image is not greater than the set second ratio, and the area of ​​the next display layer is at least the first multiple of the area of ​​the top display layer, it is considered that the user is interested in the next display layer, and the next display layer can be used as the target display layer; it should be noted that the specific values ​​of the second ratio and the first multiple can be defined as the case may be, and the embodiment of the present application is not limited. As an example, the second ratio is, for example, 20%, and the first multiple is, for example, 2. For example, when the area ratio of the top display layer occupying the screen image is not greater than 20%, and the area of ​​the next display layer is at least twice the area of ​​the top display layer, it is considered that the user is interested in the first display layer;

[0098] Example, combined Figure 5As shown, window 1 is the top-level window, window 2 is the next window, window 1 occupies a smaller area of ​​the screen image, and the area of ​​window 2 is much larger than window 1. In order to reduce the misjudgment of the user's area of ​​interest, the embodiment of the present application can expand the area of ​​interest to the user based on window 1 (the top-level display layer), and believe that the user is also interested in window 2. Therefore, the embodiment of the present application can use the display area corresponding to window 1 and window 2 as the first display area.

[0099] Case 3: The embodiment of the present application can determine whether the user is interested in the next display layer of the top display layer by determining whether the top display layer is in the main position of the screen image. If it is determined that the top display layer is not in the main position of the screen image, it can be determined that the user is interested in the next display layer of the top display layer. It should be noted that the user generally places the display layer of interest in the main position of the screen (such as the center position of the screen) to facilitate the user to perform screen operations. If the top display layer is not in the main position of the screen (such as the top display layer is not in the center position of the screen, the top display layer may be located at the upper left, lower left, upper right, lower right and other corners of the screen), it means that the user may not place the main focus on the top display layer. At this time, it is necessary to expand the area of ​​interest to the user in the screen image to reduce the misjudgment of the area of ​​interest to the user. In this case, the embodiment of the present application can use the next display layer of the top display layer as the first display area of ​​interest to the user on the basis of the top display layer. In a possible implementation, the embodiment of the present application can analyze whether the position of the top display layer is in the main position of the screen image through the display layer validity judgment (such as window validity judgment) function.

[0100] In one example, combining Figure 6 As shown, window 1 is the top-level window and window 2 is the next window. Window 1 is located in the upper left corner of the screen image instead of in the center. In order to reduce the misjudgment of the user's area of ​​interest, the embodiment of the present application can expand the area of ​​interest to the user based on window 1 (top-level display layer), and consider that the user is also interested in window 2. Therefore, the embodiment of the present application can use the display area corresponding to window 1 and window 2 as the first display area.

[0101] The above describes several situations and analysis strategies in which the user is interested in the next display layer of the top display layer. It should be noted that these situations and analysis strategies are only optional situations in which the user is interested in the next display layer. The embodiments of the present application can also support other possible situations in which the user is interested in the next display layer, which can all be regarded as the scope of protection required by the embodiments of the present application.

[0102] Of course, the next display layer of the top display layer is only an optional situation where the target display layer of the user is interested in. The embodiment of the present application can also support the situation where the target display layer is specified by the user, and can also simultaneously support the situation where the user specifies the target display layer, and analyzes whether the next display layer of the top display layer is the target display layer of the user.

[0103] The above describes a possible implementation scheme for determining the first display area for the case where there are multiple display layers in the screen image. It should be further explained that for the case where there are multiple display layers in the screen image, the embodiment of the present application may not analyze whether there is still a target display layer of interest to the user in the screen image, but directly use the display area corresponding to the top display layer as the first display area. That is, on the basis of the top display layer, further analyzing whether there is still a target display layer of interest to the user in the screen image to expand the display area of ​​interest to the user is only an optional means.

[0104] After determining the first display area of ​​interest to the user in the screen image, the range included in the first display area and the range included in the second display area may be different for different situations (the second display area is the display area in the screen image excluding the first display area), which will be explained in detail below.

[0105] In an optional implementation, when the screen image has only one display layer, the display area corresponding to the top display layer serves as the first display area, and the background area in the screen image serves as the second display area. Therefore, when the quantization coefficient of the first display area is set to be lower than the quantization coefficient of the second display area, the embodiment of the present application may set the quantization coefficient of the top display layer to be lower than the quantization coefficient of the background area. For example, if the embodiment of the present application uniformly sets a first quantization coefficient corresponding to the ROI area and a second quantization coefficient corresponding to the non-ROI area, and the first quantization coefficient is lower than the second quantization coefficient, the embodiment of the present application may set the first quantization coefficient for the display area corresponding to the top display layer and set the second quantization coefficient for the background area, so that the image content of the top display layer is encoded with the lower first quantization coefficient corresponding to the ROI area, and the image content of the background area is encoded with the higher second quantization coefficient corresponding to the non-ROI area.

[0106] In another optional implementation, when there are multiple display layers in the screen image, if only the display area corresponding to the top display layer is used as the first display area, the display area corresponding to the non-top display layer in the screen image and the background area are used as the second display area; for example, Figure 2As shown in the example, if only display layer 1 is used as the display layer of interest to the user, the display area corresponding to display layer 1 can be used as the first display area, and the display areas and background areas corresponding to display layers 2 and 3 can be used as the second display area. In view of this situation, the embodiment of the present application can use multiple methods to set the quantization coefficient:

[0107] In an optional implementation, when the first quantization coefficient corresponding to the ROI area is uniformly set, and the second quantization coefficient corresponding to the non-ROI area is uniformly set, the embodiment of the present application may set the first quantization coefficient for the display area corresponding to the top display layer, and set the second quantization coefficient for the display area corresponding to the non-top display layer and the background area; for example, Figure 2 In the example, the display layer 1 is set with a first quantization coefficient, and the display layers 2, 3 and the background area are set with a second quantization coefficient;

[0108] In other optional implementations, if the quantization coefficient of the display layer that the user is interested in is set to be lower than the quantization coefficient of the display layer that the user is not interested in, and the quantization coefficient of the display layer that the user is not interested in increases as the display level decreases (that is, for multiple display layers that the user is not interested in, the quantization coefficient and the display level are negatively correlated), then the embodiment of the present application can set the quantization coefficient of the top display layer to be lower than the quantization coefficient of the non-top display layer, and the quantization coefficient of the non-top display layer increases as the display level decreases, and at the same time, the quantization coefficient of the background area is set to be not lower than the quantization coefficient of the display layer of the lowest display level; combined with Figure 2 As shown in the example, if only display layer 1 is used as the display layer of interest to the user, then the embodiment of the present application can set the quantization coefficient of display layer 1 to be lower than the quantization coefficients of display layers 2 and 3, and the quantization coefficient of display layer 2 is lower than the quantization coefficient of display layer 3, and at the same time, the quantization display of the background area is set to be no lower than the quantization coefficient of display layer 3; it should be noted that the lower the display level of the display layer, the lower the display layer is located.

[0109] In another optional implementation, when there are multiple display layers in the screen image, if the display layer of interest to the user includes the top display layer and the target display layer outside the top display layer, the multiple display layers in the screen image can be divided into three parts: the top display layer, the target display layer and the non-target display layer that the user is not interested in. For example, Figure 2 As shown in the example, if display layer 2 is a target display layer that the user is interested in, in addition to the top display layer 1, then display layer 3 can be regarded as a non-target display layer;

[0110] Thus, the display area corresponding to the top display layer and the target display layer in the screen image can be used as the first display area, and the display area corresponding to the non-target display layer and the background area in the screen image can be used as the second display area; Figure 2For example, the first display area may be the display area corresponding to display layers 1 and 2, and the second display area may be the display area and background area corresponding to display layer 3. In view of this situation, the embodiment of the present application may use a variety of methods to set the quantization coefficient:

[0111] In an optional implementation, when the first quantization coefficient corresponding to the ROI area is uniformly set, and the second quantization coefficient corresponding to the non-ROI area is uniformly set, the embodiment of the present application can set the first quantization coefficient for the display area corresponding to the top display layer and the target display layer, and set the second quantization coefficient for the display area corresponding to the non-target display layer and the background area; for example, Figure 2 In the example, the first quantization coefficient is set for display layers 1 and 2, and the second quantization coefficient is set for display layer 3 and the background area;

[0112] In other optional implementations, if the quantization coefficient of the display layer that the user is interested in is set to be lower than the quantization coefficient of the display layer that the user is not interested in, and the quantization coefficient of the display layer that the user is not interested in increases as the display level decreases (that is, for multiple display layers that the user is not interested in, the quantization coefficient and the display level are negatively correlated), then the embodiment of the present application can set the quantization coefficients of the top display layer and the target display layer to be the same and lower than the quantization coefficient of the non-target display layer, and when there are multiple non-target display layers, the quantization coefficient of the non-target display layer increases as the display level decreases, and at the same time, the quantization coefficient of the background area is set to be not lower than the quantization coefficient of the display layer of the lowest display level; combined with Figure 2 As shown in the example, in an embodiment of the present application, the quantization coefficients of display layers 1 and 2 can be set to be the same and lower than the quantization coefficient of display layer 3. If there are other display layers below display layer 3, the quantization coefficients of display layer 3 and the display layers below it increase as the display level decreases. At the same time, the quantization display of the background area is set to be no lower than the quantization coefficient of the lowest display layer.

[0113] In other possible implementations, the screen image may have only two display layers, namely the top display layer and the next display layer. If the top display layer and the next display layer are both display layers of interest to the user, the embodiment of the present application may set the quantization coefficient of the display layer to be lower than the quantization coefficient of the background area, such as setting a first quantization coefficient corresponding to the ROI area for the display layer, and setting a second quantization coefficient corresponding to the non-ROI area for the background area.

[0114] The above describes the possible situation of the first display area and the possible situation of the second display area, and explains the possible way of setting the quantization coefficient for the first display area and the second display area, so that based on the content described above, the embodiment of the present application can obtain the first display area with a lower quantization coefficient and the second display area with a higher quantization coefficient, and then based on the quantization coefficient of the first display area and the quantization coefficient of the second display area, the screen image is encoded, and the encoded image corresponding to the screen image can be obtained, and the image encoding of the screen image is completed. Since the quantization coefficient is negatively correlated with the image quality, in the encoding process of the screen image, based on the lower quantization coefficient of the first display area, the embodiment of the present application can ensure the image quality of the first display area, and based on the higher quantization coefficient of the second display area, the embodiment of the present application can reduce the image bit rate of the second display area at the cost of a certain image quality of the second display area, so that the image data volume of the second display area is reduced, so that the embodiment of the present application can reduce the overall data volume of the encoded image and improve the image compression rate under the premise of ensuring the image quality of the first display area. At the same time, since the first display area of ​​interest to the user in the screen image is determined based on at least the top display layer, that is, the embodiment of the present application can determine the first display area of ​​interest to the user based on the display level of the display layer in the screen image, without the need to use ROI recognition technology to identify the area of ​​interest to the user in the screen image, it can greatly improve the efficiency of determining the part of the screen image that the user is interested in, implement image encoding in an efficient manner, and improve image encoding efficiency.

[0115] The image encoding method provided in the embodiment of the present application can be applied to electronic devices, such as being specifically implemented by an encoder in the electronic device. Taking the display layer as a window as an example, Figure 7 The electronic device component architecture for implementing the image encoding method provided in the embodiment of the present application is shown. Figure 7 As shown, the component architecture may include: a window system, a window manager, and an encoder;

[0116] The window system is a desktop system of any operating system, such as Windows, Linux, macOS and other operating systems. Multiple windows can be opened through the window system, such as Figure 7 In the example, window 1 and window 2 are shown. Window 1 is the top-level window, which blocks part of the area of ​​the lower-level window 2.

[0117] The window manager can obtain the display level, position information (such as the position of the boundary points of each window), resolution and other information of each window from the window system. The window manager can transparently transmit this information to the encoder so that the encoder can encode the screen image (such as the desktop image);

[0118] The encoder can obtain information such as the display level, position information, and resolution of each window from the window manager, and encode and compress the screen image based on at least the display level of each window; specifically, the encoder configures the compressed image quality for each window based on the display level of the window obtained from the window manager, and makes the window displayed in the upper layer have a higher compressed image quality; in configuring the compressed image quality, the embodiment of the present application can use quantization coefficients to implement it, such as setting a lower quantization coefficient for the window displayed in the upper layer based on the display level of the window, that is, if a higher image quality is required, a smaller quantization coefficient is configured, and if a lower image quality is required, a larger quantization coefficient is configured, thereby achieving the purpose of configuring the quantization coefficient of the area of ​​interest to the user and the quantization coefficient of the area of ​​no interest to the user based on the display level of the window;

[0119] In terms of logical implementation, the encoder may determine the display level of the window in the screen image from the window manager; determine the top-level window in the screen image according to the display level, and determine the first display area of ​​interest to the user in the screen image at least based on the top-level window; thereby setting the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image (such as the quantization coefficient of the first display area is lower than the quantization coefficient of the second display area), and the second display area is the display area outside the first display area in the screen image; and then encoding the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image;

[0120] After the encoder is processed, if the window the user is interested in is only Figure 7 Window 1 in the example can achieve the following effect: the image quality of window 1 is guaranteed, and even if the image quality of window 2 is lower, it will not have a big impact on the user, so that image encoding can be achieved while taking into account both image quality and image compression rate.

[0121] In a more specific description, the encoder may rely on the display level obtained from the window manager and combine the position information and resolution information of each window when determining the first display area of ​​interest to the user in the screen image; specifically, combining Figure 7For example, if the only window that the user is interested in is window 1, the range enclosed by the four boundary point positions (A1, B1, C1, D1) of window 1 is the first display area that the user is interested in, and the area outside the range enclosed by the four boundary point positions (A1, B1, C1, D1) of window 1 can be regarded as the second display area that the user is not interested in; the encoder configures different compressed image qualities for the first display area and the second display area, respectively, to ensure that a higher compressed image quality is configured for the first display area and a lower compressed image quality is configured for the second display area, thereby improving the compression efficiency without the user being aware of it.

[0122] In further description, the embodiment of the present application may add a window activity detection function and a window validity judgment function to the encoder to make the detection of the first display area of ​​interest to the user in the screen image more accurate. The following describes possible improvement scenarios:

[0123] 1. Based on the window activity detection function, if it is detected that window 2 displays a video or animation, that is, the picture of window 2 has a large change in the time domain, and based on the window validity judgment function, if the proportion of the unobstructed area of ​​the detection window 1 is not less than the first proportion (such as 60%), then the placement of window 1 and window 2 may be intentional by the user. At this time, in order to reduce the misjudgment of the first display area, the encoder can expand the range of the first display area of ​​interest to the user on the basis of the top-level window 1, so that the first display area includes the display areas of window 1 and window 2, such as setting the first display area to be the range enclosed by the four boundary points (A2, B2, C2, D2) of window 2 and the four boundary points (A1, B1, C1, D1) of window 1;

[0124] 2. Based on the window validity judgment function, if it is judged that the area of ​​window 1 occupies a smaller proportion of the screen than the second proportion (such as 20%), and the area of ​​window 2 is not less than the area of ​​window 1 at the first multiple (such as 2 times, etc.), then since the area of ​​the top-level window 1 is smaller, from the perspective of perception, the user generally will not focus on window 1 after scaling the top-level window 1. Therefore, in order to reduce the misjudgment of the first display area, the encoder may expand the range of the first display area of ​​interest to the user on the basis of the top-level window 1, so that the first display area includes the display areas of window 1 and window 2, such as setting the first display area to be the area enclosed by the four boundary points (A2, B2, C2, D2) of window 2 and the four boundary points (A1, B1, C1, D1) of window 1;

[0125] 3. Based on the window validity judgment function, if it is judged that window 1 is not in the center position of the screen image, for example, window 1 is in the upper left, lower left, upper right, lower right and other corners of the screen, then from the perspective of perception, users generally do not focus on these corners. Therefore, in order to reduce the misjudgment of the first display area, the encoder may expand the range of the first display area of ​​interest to the user on the basis of the top-level window 1, so that the first display area includes the display areas of window 1 and window 2, such as setting the first display area to the range enclosed by the four boundary points (A2, B2, C2, D2) of window 2 and the four boundary points (A1, B1, C1, D1) of window 1.

[0126] The above describes possible ways in which the encoder further determines the target display layer of interest to the user based on the top display layer to determine the first display area of ​​interest to the user. These methods are merely optional means for the encoder to further improve the first display area of ​​interest to the user based on the top display layer. Based on the foregoing description, the encoder can fully adopt other possible alternative means, which will not be further elaborated here.

[0127] It should be noted that after the encoder completes image encoding, the encoded image may be further processed, such as transmitting the encoded image to other devices, which may decode the image and display the decoded image.

[0128] In an embodiment of the present application, the encoder can identify the display level of the window through the window manager, and thus determine the first display area of ​​interest to the user in the screen image at least based on the display level of the window, and then set different compressed image qualities for the first display area of ​​interest to the user and the second display area of ​​no interest to the user, thereby implementing the encoding of the screen image; compared with the image encoding scheme based on ROI recognition, the embodiment of the present application can determine the first display area of ​​interest to the user based on the display level of the window in the screen image in a multi-window display environment, without the need to use ROI recognition technology to identify the area of ​​interest to the user in the screen image, thereby greatly improving the efficiency of determining the part of the screen image that the user is interested in, and implementing image encoding in an efficient manner, and by setting different compression configurations for the first display area and the second display area, the embodiment of the present application can reduce the overall data volume of the encoded image and improve the image compression rate while ensuring the image quality of the first display area in the screen image, thereby improving the image encoding efficiency while taking into account both image quality and image compression efficiency.

[0129] For the image encoding method provided in the embodiment of the present application, the embodiment of the present application also provides a corresponding image decoding method, which can be applied to electronic devices; taking the screen sharing service as an example, the electronic device applying the image decoding method can, for example, obtain the shared screen image of the electronic device; taking the cloud screen service as an example, the electronic device applying the image decoding method can, for example, use the terminal device of the cloud screen service; in an optional implementation, Figure 8 The optional process of the image decoding method provided by the embodiment of the present application is shown as follows: Figure 8 As shown, the process may include:

[0130] Step S300, obtain the encoded image corresponding to the screen image, wherein the compressed image quality of the first display area in which the user is interested in the encoded image is higher than the compressed image quality of the second display area, and the second display area is a display area outside the first display area, and the first display area includes at least the top display layer in the screen image.

[0131] Step S310: Decode the encoded image.

[0132] The image encoding process of the embodiment of the present application is based on the compressed image quality of the first display area being higher than the compressed image quality of the second display area. Therefore, after image decoding, the image quality of the first display area in the decoded image is better than the image quality of the second display area, that is, the clarity and image quality of the first display area that the user is interested in can be guaranteed. Since image decoding and image encoding are mutually inverse processes, the embodiment of the present application will not be further expanded here.

[0133] Further, as a possible usage scenario, in the image encoding process, the display layer in the screen image referred to in the embodiment of the present application may be multiple windows opened by different applications (applications), and / or at least one window opened by an application, etc., and its possible usage scenarios are, for example:

[0134] The user opens multiple application windows with the desktop as the background. One application may have at least one window opened. These windows will be displayed in layers with the desktop as the background. When the user needs to encode the desktop image, the user can use the image encoding method provided in the embodiment of the present application to complete the encoding of the desktop image.

[0135] When a user opens multiple windows in an application, the multiple windows may be displayed in a hierarchical manner on the application interface of the application. When the user needs to encode the application interface image, the user can use the image encoding method provided in the embodiment of the present application to complete the encoding of the application interface image.

[0136] From the perspective of service deployment, the image encoding method provided in the embodiment of the present application can be deployed in a screen sharing service (such as a desktop sharing service, a remote desktop service, etc.), or in a cloud screen service (such as a cloud desktop service, etc.). The following takes the desktop sharing service as an example to illustrate the image encoding scheme provided in the embodiment of the present application.

[0137] like Fig. 9 As shown, terminal A needs to share the desktop image with terminal B through a desktop sharing service (such as a network that provides desktop sharing services, etc.), then terminal A needs to encode each frame of the desktop image to form a video encoding image, and then transmit the video encoding image to terminal B, so that terminal B can display the desktop image shared by terminal A;

[0138] For any frame of desktop image that terminal A needs to encode, the encoder of terminal A can determine the display level of the display layer (such as a window) in the desktop, determine the top display layer in the desktop, and further analyze whether there is a target display layer of interest to the user in the display layer of the desktop in addition to the top display layer; if so, the display area corresponding to the top display layer and the analyzed target display layer is used as the first display area of ​​interest to the user; if not, the display area corresponding to the top display layer is directly used as the first display area of ​​interest to the user; further, the encoder can set the quantization coefficient of the first display area to be lower than the quantization coefficient of the second display area in addition to the first display area, and encode the desktop image according to the quantization coefficient of the first display area and the quantization coefficient of the second display area to obtain an encoded image; thus, each frame of the encoded image obtained by the encoder can be combined to form a video encoded image, which is shared to terminal B through the desktop sharing service;

[0139] During the above processing of terminal A, terminal A can reduce the overall data volume after desktop image encoding and improve image encoding efficiency while ensuring the image quality of the first display area of ​​the desktop that the user is interested in. It also has the advantage of saving the transmission bandwidth of video encoded images, and can achieve reliable and efficient desktop sharing services.

[0140] The above describes multiple implementation schemes provided by the embodiments of the present application. The various optional methods introduced in each implementation scheme can be combined and cross-referenced with each other without conflict, thereby extending a variety of possible implementation schemes, which can all be considered as implementation schemes disclosed and open in the embodiments of the present application.

[0141] The image encoding device provided in the embodiment of the present application is introduced below. The image encoding device described below can be considered as an electronic device, which is a functional module required to implement the image encoding method provided in the embodiment of the present application. The content of the image encoding device described below can be referenced to the content of the image encoding method described above.

[0142] In an optional implementation, Fig.10 An optional block diagram of an image encoding device provided in an embodiment of the present application is shown, referring to Fig.10 As shown, the image encoding device may include:

[0143] A display level determination module 100, used to determine the display level of a display layer in a screen image;

[0144] An interest region determining module 110, configured to determine a top display layer in the screen image according to the display hierarchy, and determine a first display region in the screen image that is of interest to the user based at least on the top display layer;

[0145] A compression setting module 120, configured to set a compression image quality of the first display area to be higher than a compression image quality of a second display area in the screen image, the second display area being a display area outside the first display area in the screen image;

[0146] The encoding module 130 is used to encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image.

[0147] Optionally, there are multiple display layers in the screen image; the interest area determination module 110, configured to determine a first display area of ​​interest to the user in the screen image based at least on the top display layer, includes:

[0148] Determine whether, in addition to the top display layer, there is a target display layer of interest to the user in the screen image;

[0149] If there is still a target display layer that the user is interested in in the screen image, the display area corresponding to the top display layer and the target display layer is determined as the first display area.

[0150] Optionally, the interest region determination module 110 is configured to determine whether, in addition to the top display layer, there is a target display layer in the screen image that the user is interested in, including:

[0151] Determine whether the user is interested in a display layer next to the top display layer, the next display layer being a display layer one level lower than the top display layer;

[0152] Correspondingly, the interest region determining module 110 is configured to determine the display region corresponding to the top display layer and the target display layer as the first display region if there is a target display layer of interest to the user in the screen image, including:

[0153] If the user is interested in the next display layer, the display area corresponding to the top display layer and the next display layer is determined as the first display area.

[0154] Optionally, on one hand, the interest area determination module 110, for determining whether the user is interested in the next display layer of the top display layer, comprises:

[0155] Whether the user is interested in the next display layer is determined at least according to the display content of the next display layer.

[0156] Optionally, the interest region determination module 110 is configured to determine whether the user is interested in the next display layer at least according to the display content of the next display layer, including:

[0157] Determine whether the next display layer displays dynamic content and whether the proportion of the unobstructed area of ​​the next display layer is not less than a set first proportion; wherein, if it is determined that the next display layer displays dynamic content and the proportion of the unobstructed area of ​​the next display layer is not less than the set first proportion, the user is interested in the next display layer.

[0158] Optionally, on the other hand, the interest area determination module 110, for determining whether the user is interested in the next display layer of the top display layer, comprises:

[0159] Whether the user is interested in the next display layer is determined according to the area of ​​the top display layer and the area relationship between the top display layer and the next display layer.

[0160] Optionally, the interest region determination module 110 is configured to determine whether the user is interested in the next display layer of the top display layer according to the area of ​​the top display layer and the area relationship between the top display layer and the next display layer, including:

[0161] Determine whether the area ratio of the screen image occupied by the top-level display layer is not greater than the set second ratio, and whether the area of ​​the next display layer is at least the first multiple of the area of ​​the top-level display layer; wherein, if it is determined that the area ratio of the screen image occupied by the top-level display layer is not greater than the set second ratio, and the area of ​​the next display layer is at least the first multiple of the area of ​​the top-level display layer, the user is interested in the next display layer.

[0162] Optionally, in another aspect, the interest region determination module 110 is configured to determine whether the user is interested in the next display layer of the top display layer, including:

[0163] It is determined whether the top display layer is in the main position of the screen image, wherein if it is determined that the top display layer is not in the main position of the screen image, the user is interested in the next display layer.

[0164] Optional, Fig.11 Another optional block diagram of the image encoding device provided in the embodiment of the present application is shown, Fig.10 and Fig.11 As shown, the device may also include:

[0165] A position determination module 140, configured to determine position information of each display layer in the screen image, wherein the position information of each display layer includes the position of a boundary point of the display layer;

[0166] Accordingly, the interest region determining module 110 is configured to determine the display regions corresponding to the top display layer and the next display layer as the first display region, including:

[0167] A display area corresponding to the boundary point position of the top display layer and the boundary point position of the next display layer as a whole is determined as the first display area.

[0168] Optionally, the interest region determination module 110 is configured to determine whether, in addition to the top display layer, there is a target display layer in the screen image that the user is interested in, including:

[0169] Determine whether there is a target display layer with a tag of interest added in the screen image except the top display layer;

[0170] Correspondingly, the interest region determining module 110 is configured to determine the display region corresponding to the top display layer and the target display layer as the first display region if there is a target display layer of interest to the user in the screen image, including:

[0171] If there is a target display layer with an interest tag added in the screen image, the display area corresponding to the top display layer and the target display layer with the interest tag added is determined as the first display area.

[0172] Optionally, there are one or more display layers in the screen image. In a possible alternative implementation, the interest region determining module 110 is configured to determine a first display region of interest to the user in the screen image based at least on the top display layer, including:

[0173] The display area corresponding to the top display layer is determined as the first display area.

[0174] Optionally, when there is one display layer in the screen image, the second display area includes the background area of ​​the screen image excluding the display layer; when there are multiple display layers in the screen image, the second display area includes display areas corresponding to other display layers excluding the top display layer, and the background area of ​​the screen image excluding the display layer.

[0175] Optionally, if there are multiple display layers in the screen image, in one case, the multiple display layers in the screen image include: the top display layer, a target display layer of interest to the user other than the top display layer, and a non-target display layer of no interest to the user; the first display area includes a display area corresponding to the top display layer and a display area corresponding to the target display layer, and the second display area includes: a display area corresponding to the non-target display layer, and a background area in the screen image other than the display layers.

[0176] Optionally, after determining the first display area and the second display area in the screen image, the compression setting module 120, configured to set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, may include:

[0177] Setting a quantization coefficient of the first display area to be lower than a quantization coefficient of a second display area in the screen image, wherein the quantization coefficient is negatively correlated with the quality of the compressed image;

[0178] Correspondingly, the encoding module 130 is used to encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area, and the obtained encoded image includes:

[0179] The screen image is encoded according to the quantization coefficient of the first display area and the quantization coefficient of the second display area to obtain an encoded image.

[0180] Optionally, the compression setting module 120 is used to set a quantization coefficient of the first display area, which is lower than the quantization coefficient of the second display area in the screen image, including:

[0181] A first quantization coefficient corresponding to the ROI is set for the first display area, and a second quantization coefficient corresponding to the non-ROI is set for the second display area, wherein the first quantization coefficient is lower than the second quantization coefficient.

[0182] Optionally, if the multiple display layers in the screen image include: the top display layer, a target display layer that the user is interested in except the top display layer, and a non-target display layer that the user is not interested in; then in an optional implementation, the compression setting module 120 is used to set the quantization coefficient of the first display area, which is lower than the quantization coefficient of the second display area in the screen image, including:

[0183] The quantization coefficients of the top display layer and the target display layer are set to be the same and lower than the quantization coefficient of the non-target display layer, and when there are multiple non-target display layers, the quantization coefficient of the non-target display layer increases as the display level decreases; and the quantization coefficient of the background area is set to be no lower than the quantization coefficient of the non-target display layer of the lowest display level.

[0184] Optionally, in one usage scenario, the display layer in the screen image includes: multiple windows opened by different applications, wherein one application opens at least one window, and the multiple windows are displayed in layers and superimposed on each other with the desktop as the background; in another usage scenario, the display layer in the screen image includes: multiple windows opened within one application, and the multiple windows are displayed in layers and superimposed on each other on the application interface of the application.

[0185] Optionally, as an application, the electronic device that performs image encoding may be a terminal device, and the screen image is a local screen image; further, Fig.12 Another optional block diagram of the image encoding device provided by the embodiment of the present application is shown, Fig.10 and Fig.12 As shown, the device may also include:

[0186] A screen sharing module 150, configured to share the encoded image with other terminal devices participating in a screen sharing service;

[0187] Correspondingly, as an optional implementation, the local screen image is a local desktop image; and the screen sharing service includes a desktop sharing service or a remote desktop service.

[0188] Optionally, as another application, the electronic device that performs image encoding may be a cloud service device that provides a cloud screen service, and the screen image is a cloud screen image; further, Fig.13 Another optional block diagram of the image encoding device provided by the embodiment of the present application is shown, combined with Fig.10 and Fig.13 As shown, the device may also include:

[0189] A cloud screen transmission module 160, used to transmit the encoded image to a terminal device using a cloud screen service;

[0190] Accordingly, in an optional implementation, the cloud screen image includes a cloud desktop image, and the cloud screen service includes a cloud desktop service.

[0191] In an optional implementation, the function of the image encoding device provided in the embodiment of the present application can be implemented in an encoder of an electronic device (which may be a terminal device or a cloud service device). Specifically, Figure 7 As shown, the image encoding device may include: an encoder;

[0192] The encoder is used to: determine a display level of a window in a screen image; determine a top-level window in the screen image according to the display level, and determine a first display area of ​​interest to a user in the screen image at least based on the top-level window; set a compressed image quality of the first display area to be higher than a compressed image quality of a second display area in the screen image, the second display area being a display area outside the first display area in the screen image; and encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image.

[0193] Optionally, the image encoding device may further include:

[0194] The window system used to open the window;

[0195] A window manager for acquiring at least a display level of each window from the window system, wherein the window manager transmits at least the acquired display level to the encoder.

[0196] The image encoding device provided in the embodiment of the present application can improve the image encoding efficiency while taking into account both image quality and image compression efficiency; accordingly, the electronic device that performs image encoding (which may be a terminal device or a cloud service device) can load the above-mentioned image encoding device to implement the image encoding method provided in the embodiment of the present application.

[0197] As an optional implementation, Fig.14 An optional block diagram of an electronic device provided by an embodiment of the present application is shown, including at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0198] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0199] Optionally, the communication interface 2 may be an interface of a communication module for performing network communication;

[0200] Optionally, processor 1 may be a CPU (central processing unit), a GPU (Graphics Processing Unit), an NPU (embedded neural network processor), an FPGA (Field Programmable Gate Array), a TPU (tensor processing unit), an AI chip, an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement an embodiment of the present application.

[0201] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0202] The memory 3 stores one or more computer executable instructions, and the processor 1 calls the one or more computer executable instructions to execute the image encoding method provided in the embodiment of the present application.

[0203] An embodiment of the present application also provides a storage medium, which stores one or more computer-executable instructions. When the one or more computer-executable instructions are executed, the image encoding method provided by the embodiment of the present application is implemented.

[0204] An embodiment of the present application also provides an image decoding device, which can be considered as an electronic device, and is a functional module required to implement the image decoding method provided in the embodiment of the present application; the content of the image decoding device described below can be referenced in correspondence with the content described above.

[0205] In an optional implementation, Fig.15 An optional block diagram of an image decoding device provided in an embodiment of the present application is shown, referring to Fig.15 As shown, the image decoding device may include:

[0206] The coded image acquisition module 200 is used to acquire a coded image corresponding to a screen image, wherein the compressed image quality of a first display area of ​​interest to a user in the coded image is higher than the compressed image quality of a second display area, the second display area being a display area outside the first display area, and the first display area at least including a top display layer in the screen image;

[0207] The decoding module 210 is configured to decode the encoded image.

[0208] In an optional implementation, the functions of the image decoding device provided in the embodiment of the present application can be implemented in a decoder of an electronic device (which may be a terminal device or a cloud service device). Specifically, the image decoding device may include: a decoder; the decoder is used to: obtain an encoded image corresponding to a screen image, the compressed image quality of a first display area of ​​interest to a user in the encoded image is higher than the compressed image quality of a second display area, the second display area is a display area outside the first display area, and the first display area includes at least the top-level window in the screen image; and decode the encoded image.

[0209] The decoding device provided in the embodiment of the present application has a decoding result in which the image quality of the first display area is better than the image quality of the second display area, that is, the clarity and image quality of the first display area of ​​interest to the user can be guaranteed. Accordingly, the electronic device that performs image decoding (which may be a terminal device or a cloud service device) can implement the image decoding method provided in the embodiment of the present application by loading the image decoding device described above. Optionally, the block diagram of the electronic device that performs image decoding can be combined with reference to Fig.14 As shown, it includes at least one memory and at least one processor, the memory stores one or more computer executable instructions, and the processor calls the one or more computer executable instructions to execute the image decoding method provided by the embodiment of the present application.

[0210] An embodiment of the present application also provides a storage medium, which stores one or more computer-executable instructions. When the one or more computer-executable instructions are executed, the image decoding method provided by the embodiment of the present application is implemented.

[0211] Although the embodiments of the present application are disclosed above, the present application is not limited thereto. Any person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined by the claims.

Claims

1. A method for encoding an image, wherein: Applied to electronic equipment, the method comprises: Determine the display level of the display layer in the screen image; Determine a top display layer in the screen image according to the display hierarchy, and determine a first display area of ​​interest to a user in the screen image at least based on the top display layer; Setting the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, where the second display area is a display area outside the first display area in the screen image; encoding the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image; Among them, there are multiple display layers in the screen image; determining the first display area of ​​interest to the user in the screen image at least based on the top display layer includes: judging whether there is a target display layer of interest to the user in the screen image in addition to the top display layer; if there is a target display layer of interest to the user in the screen image, determining the display area corresponding to the top display layer and the target display layer as the first display area.

2. The image encoding method according to claim 1, wherein: The determining whether there is a target display layer of interest to the user in the screen image in addition to the top display layer includes: Determine whether the user is interested in a display layer next to the top display layer, the next display layer being a display layer one level lower than the top display layer; If there is a target display layer that the user is interested in in the screen image, determining the display area corresponding to the top display layer and the target display layer as the first display area includes: If the user is interested in the next display layer, the display area corresponding to the top display layer and the next display layer is determined as the first display area.

3. The image encoding method according to claim 2, wherein: The step of determining whether the user is interested in the next display layer of the top display layer comprises: Whether the user is interested in the next display layer is determined at least according to the display content of the next display layer.

4. The image encoding method according to claim 3, wherein: The determining whether the user is interested in the next display layer at least according to the display content of the next display layer comprises: Determine whether the next display layer displays dynamic content and whether the proportion of the unobstructed area of ​​the next display layer is not less than a set first proportion; wherein, if it is determined that the next display layer displays dynamic content and the proportion of the unobstructed area of ​​the next display layer is not less than the set first proportion, the user is interested in the next display layer.

5. The image encoding method according to claim 2, wherein: The step of determining whether the user is interested in the next display layer of the top display layer comprises: Whether the user is interested in the next display layer is determined according to the area of ​​the top display layer and the area relationship between the top display layer and the next display layer.

6. The image encoding method according to claim 5, wherein: The determining, based on the area of ​​the top display layer and the area relationship between the top display layer and the next display layer, whether the user is interested in the next display layer comprises: Determine whether the area ratio of the screen image occupied by the top-level display layer is not greater than the set second ratio, and whether the area of ​​the next display layer is at least the first multiple of the area of ​​the top-level display layer; wherein, if it is determined that the area ratio of the screen image occupied by the top-level display layer is not greater than the set second ratio, and the area of ​​the next display layer is at least the first multiple of the area of ​​the top-level display layer, the user is interested in the next display layer.

7. The image encoding method according to claim 2, wherein: The step of determining whether the user is interested in the next display layer of the top display layer comprises: It is determined whether the top display layer is in the main position of the screen image, wherein if it is determined that the top display layer is not in the main position of the screen image, the user is interested in the next display layer.

8. The image encoding method according to claim 2, wherein: Also includes: Determine position information of each display layer in the screen image, wherein the position information of the display layer includes the position of the boundary point of the display layer; The step of determining the display areas corresponding to the top display layer and the next display layer as the first display area includes: A display area corresponding to the boundary point position of the top display layer and the boundary point position of the next display layer as a whole is determined as the first display area.

9. The image encoding method according to claim 1, wherein: The determining whether there is a target display layer of interest to the user in the screen image in addition to the top display layer includes: Determine whether there is a target display layer with a tag of interest added in the screen image except the top display layer; If there is a target display layer that the user is interested in in the screen image, determining the display area corresponding to the top display layer and the target display layer as the first display area includes: If there is a target display layer with an interest tag added in the screen image, the display area corresponding to the top display layer and the target display layer with the interest tag added is determined as the first display area.

10. The image encoding method according to claim 1, wherein: The display layer in the screen image is one or more; and determining the first display area of ​​interest to the user in the screen image at least based on the top display layer comprises: The display area corresponding to the top display layer is determined as the first display area.

11. The image encoding method according to claim 10, wherein: When there is one display layer in the screen image, the second display area includes the background area of ​​the screen image excluding the display layer; when there are multiple display layers in the screen image, the second display area includes the display areas corresponding to the other display layers excluding the top display layer, and the background area of ​​the screen image excluding the display layer.

12. The image encoding method according to claim 1, wherein: The multiple display layers in the screen image include: the top display layer, the target display layer of interest to the user other than the top display layer, and the non-target display layer of no interest to the user; the first display area includes the display area corresponding to the top display layer and the display area corresponding to the target display layer, and the second display area includes: the display area corresponding to the non-target display layer, and the background area in the screen image other than the display layers.

13. The image encoding method according to any one of claims 1 to 12, wherein: The setting of the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image comprises: Setting a quantization coefficient of the first display area to be lower than a quantization coefficient of a second display area in the screen image, wherein the quantization coefficient is negatively correlated with the quality of the compressed image; The step of encoding the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain the encoded image comprises: The screen image is encoded according to the quantization coefficient of the first display area and the quantization coefficient of the second display area to obtain an encoded image.

14. The image encoding method according to claim 13, wherein: The step of setting the quantization coefficient of the first display area to be lower than the quantization coefficient of the second display area in the screen image comprises: A first quantization coefficient corresponding to a region of interest ROI is set for the first display area, and a second quantization coefficient corresponding to a non-ROI is set for the second display area, wherein the first quantization coefficient is lower than the second quantization coefficient.

15. The image encoding method according to claim 13, wherein: The multiple display layers in the screen image include: The top display layer, a target display layer other than the top display layer that the user is interested in, and a non-target display layer that the user is not interested in; The step of setting the quantization coefficient of the first display area to be lower than the quantization coefficient of the second display area in the screen image comprises: The quantization coefficients of the top display layer and the target display layer are set to be the same and lower than the quantization coefficient of the non-target display layer, and when there are multiple non-target display layers, the quantization coefficient of the non-target display layer increases as the display level decreases; and the quantization coefficient of the background area is set to be no lower than the quantization coefficient of the non-target display layer of the lowest display level.

16. The image encoding method according to claim 1, wherein: The display layer in the screen image includes: multiple windows opened by different applications, wherein one application opens at least one window, and the multiple windows are displayed in layers with the desktop as the background; Alternatively, the display layer in the screen image includes: a plurality of windows opened in an application, and the plurality of windows are displayed in a hierarchical manner in an application interface of the application.

17. The image encoding method according to claim 1, wherein: The electronic device is a terminal device; the screen image is a local screen image.

18. The image encoding method according to claim 17, wherein: The method further comprises: The encoded image is shared with other terminal devices participating in the screen sharing service.

19. The image encoding method according to claim 18, wherein: The local screen image is a local desktop image; The screen sharing service includes a desktop sharing service or a remote desktop service.

20. The image encoding method according to claim 1, wherein: The electronic device is a cloud service device that provides a cloud screen service, and the screen image is a cloud screen image; the method further includes: The encoded image is transmitted to a terminal device using a cloud screen service.

21. The image encoding method according to claim 20, wherein: The cloud screen image includes a cloud desktop image, and the cloud screen service includes a cloud desktop service.

22. An image decoding method, wherein: Applied to electronic equipment, the method comprises: Acquire a coded image corresponding to a screen image, wherein the screen image has multiple display layers, and the compressed image quality of a first display area of ​​interest to a user in the coded image is higher than the compressed image quality of a second display area, the second display area is a display area outside the first display area, the first display area includes at least a top display layer and a target display layer in the screen image, and the target display layer is a display layer of interest to the user existing in the screen image except the top display layer; The encoded image is decoded.

23. An image encoding device, wherein: include: A display level determination module, used to determine the display level of the display layer in the screen image; an interest region determining module, configured to determine a top display layer in the screen image according to the display hierarchy, and to determine a first display region in the screen image that is of interest to the user based at least on the top display layer; A compression setting module, used to set the compressed image quality of the first display area to be higher than the compressed image quality of the second display area in the screen image, where the second display area is a display area outside the first display area in the screen image; an encoding module, configured to encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image; Among them, there are multiple display layers in the screen image; the interest area determination module is also used to: determine whether there is a target display layer that the user is interested in in the screen image in addition to the top display layer; if there is a target display layer that the user is interested in in the screen image, the display area corresponding to the top display layer and the target display layer is determined as the first display area.

24. An image encoding device, wherein: include: Encoder; The encoder is used to: Determines the display level of the window in the screen image; Determine a top-level window in the screen image according to the display level, and determine a first display area of ​​interest to a user in the screen image at least based on the top-level window; set a compressed image quality of the first display area to be higher than a compressed image quality of a second display area in the screen image, the second display area being a display area outside the first display area in the screen image; encode the screen image according to the compressed image quality of the first display area and the compressed image quality of the second display area to obtain an encoded image; Among them, there are multiple display layers in the screen image; determining the first display area of ​​interest to the user in the screen image based on the top-level window includes: judging whether there is a target display layer of interest to the user in the screen image in addition to the top-level window; if there is a target display layer of interest to the user in the screen image, determining the display area corresponding to the top-level window and the target display layer as the first display area.

25. The image encoding device according to claim 24, wherein: Also included: a window system for opening windows; A window manager for acquiring at least a display level of each window from the window system, wherein the window manager transmits at least the acquired display level to the encoder.

26. An image decoding device, wherein: include: a coded image acquisition module, configured to acquire a coded image corresponding to a screen image, wherein the screen image has multiple display layers, and the compressed image quality of a first display area of ​​interest to a user in the coded image is higher than the compressed image quality of a second display area, the second display area is a display area outside the first display area, the first display area includes at least a top display layer and a target display layer in the screen image, and the target display layer is a display layer of interest to the user existing in the screen image except the top display layer; A decoding module is used to decode the encoded image.

27. An electronic device, wherein: It includes at least one memory and at least one processor; the memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to execute the image encoding method as described in any one of claims 1 to 21, or the image decoding method as described in claim 22.

28. A storage medium, wherein: The storage medium stores one or more computer-executable instructions, and when the one or more computer-executable instructions are executed, the image encoding method according to any one of claims 1 to 21, or the image decoding method according to claim 22 is implemented.

Images