Training method of image compression model and image compression method and device

Abstract

The embodiment of the invention provides a training method of an image compression model and an image compression method and device.The training method of the image compression model comprises the steps that a sample image is obtained, the sample image is divided to obtain region information, and the region information is used for indicating at least two regions included in the sample image; inputting the sample image and the regional information into the initial image compression model to obtain a corresponding predicted reconstructed image; according to the predicted reconstructed image and the sample image, the loss functions corresponding to the at least two areas are adopted to determine model reconstruction loss, the initial image compression model is trained according to the model reconstruction loss, a trained target image compression model is obtained, and the loss functions corresponding to the at least two areas are different. A sample image is divided into at least two regions, different loss functions are adopted for different regions to determine model reconstruction loss, and the characteristics of different loss functions are utilized to constrain image reconstruction targets of different regions, so that the compression effect of different regions is relatively good.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of image processing technology, and in particular to a training method for an image compression model, an image compression method and a device. Background Art

[0002] With the rapid development of computer technology, communication technology and network technology, various videos and images are widely used as important information. Image compression is of great significance for data storage and transmission. Uncompressed images will take up a huge storage space and bring great pressure to transmission. High-quality compressed images play a vital role in network transmission and storage on mobile platforms.

[0003] In the existing technology, image compression technology based on deep learning has developed rapidly in recent years. Image compression models can be used to achieve image compression. Although the method based on deep learning has made certain progress, there are still problems such as image blur, noise, compression artifacts, etc., and the image compression effect is poor. Therefore, a better image compression solution is urgently needed. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a training method for an image compression model. One or more embodiments of this specification also relate to an image compression method, a training device for an image compression model, an image compression device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a method for training an image compression model is provided, comprising:

[0006] Acquire a sample image, and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image;

[0007] Inputting the sample image and the region information into an initial image compression model to obtain a corresponding predicted reconstructed image;

[0008] According to the predicted reconstructed image and the sample image, the model reconstruction loss is determined using the loss functions corresponding to the at least two regions, and the initial image compression model is trained according to the model reconstruction loss to obtain a trained target image compression model, wherein the loss functions corresponding to the at least two regions are different.

[0009] According to a second aspect of the embodiments of this specification, there is provided an image compression method, comprising:

[0010] Obtain the image to be compressed;

[0011] The image to be compressed is input into the target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is obtained by training using the above-mentioned image compression model training method.

[0012] According to a third aspect of an embodiment of this specification, a training device for an image compression model is provided, comprising:

[0013] A division module is configured to obtain a sample image and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image;

[0014] A first obtaining module is configured to input the sample image and the region information into an initial image compression model to obtain a corresponding predicted reconstructed image;

[0015] The training module is configured to determine the model reconstruction loss based on the predicted reconstructed image and the sample image using the loss functions corresponding to the at least two regions, train the initial image compression model based on the model reconstruction loss, and obtain a trained target image compression model, wherein the loss functions corresponding to the at least two regions are different.

[0016] According to a fourth aspect of the embodiments of this specification, there is provided an image compression device, including:

[0017] An acquisition module, configured to acquire an image to be compressed;

[0018] The second acquisition module is configured to input the image to be compressed into the target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is obtained by training using the above-mentioned image compression model training method.

[0019] According to a fifth aspect of the embodiments of this specification, a computing device is provided, including:

[0020] Memory and processor;

[0021] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the training method of the above-mentioned image compression model or the steps of the image compression method are implemented.

[0022] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the training method of the above-mentioned image compression model or the steps of the image compression method.

[0023] According to the seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the training method of the above-mentioned image compression model or the steps of the image compression method.

[0024] An embodiment of the present specification realizes that a sample image is divided into at least two regions, the region information of the at least two regions and the sample image are input into the initial image compression model together, the corresponding predicted reconstructed image is obtained, the loss function corresponding to the at least two regions is used, the loss between the predicted reconstructed image and the sample image is calculated, the model reconstruction loss is determined, the model is trained, and the trained target image compression model is obtained, and the loss functions corresponding to the at least two regions are different. In this way, the sample image is divided into at least two regions, and different loss functions are used in different regions to determine the model reconstruction loss, so as to make full use of the characteristics of different loss functions to constrain the image reconstruction targets of different regions, that is, different regions use different loss functions to constrain the generation of reconstructed images, so that the target image compression model obtained by training can use different encoding methods and allocate different bit rates for different regions, so as to ensure better compression effects in different regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of a processing process of an image compression method based on deep learning provided by an embodiment of this specification;

[0026] Figure 2 is a flowchart of a training method for an image compression model provided by an embodiment of this specification;

[0027] Figure 3 is a flow chart of an image compression method provided by an embodiment of this specification;

[0028] Figure 4 is a processing flow chart of an image compression method provided by an embodiment of this specification;

[0029] Figure 5 It is a structural schematic diagram of a training device for an image compression model provided by an embodiment of this specification;

[0030] Figure 6 is a structural schematic diagram of an image compression device provided by an embodiment of this specification;

[0031] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0032] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0033] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] First, the terms involved in one or more embodiments of this specification are explained.

[0037] Image compression: is a data processing technique that aims to reduce the size of image files while preserving their visual quality as much as possible. This technique is critical for storing and transmitting digital images because it can save storage space, speed up network transmission and reduce bandwidth usage. Image compression is mainly divided into two types: lossless compression and lossy compression.

[0038] Deep learning: It is a subfield of machine learning. It imitates the working method of the human brain and uses a multi-layer neural network model to learn high-level abstract representations of data. In recent years, this technology has achieved remarkable success in many fields such as image recognition, speech recognition, natural language processing, and recommendation systems, and has become an important driving force for the development of artificial intelligence.

[0039] MSE (Mean Squared Error): is a commonly used metric for evaluating the error of a prediction model. It is mainly used to measure the degree of difference between the predicted value and the actual value. Especially in regression analysis problems, it measures the average of the sum of the squares of the differences between the model predicted value and the actual observed value. This design takes into account both the size of the error and the direction of the error. The larger the error, the larger its square will be, thus occupying a larger proportion in the overall evaluation.

[0040] MS-SSIM (Multi-Scale Structural Similarity): is an index used to evaluate the similarity between two images. It is a multi-scale extended version of SSIM (Structural Similarity Index). SSIM itself is an improved method for evaluating image quality. Compared with the traditional mean square error (MSE), it focuses more on maintaining the consistency of image structure and visual quality. MS-SSIM further introduces the concept of multi-scale on this basis, and more comprehensively considers the structural similarity of images at different resolutions.

[0041] Generative models are an important type of model in machine learning. Their main goal is to learn the underlying data distribution patterns from existing data sets, and then generate new and similar data samples. Generative models can not only be used for data generation, but also help us understand and explore the intrinsic structure of data, and perform tasks such as data enhancement, missing value filling, and density estimation.

[0042] Generative compression: It is a technology that uses generative models to compress data. Unlike traditional compression algorithms (such as JPEG, H.264, etc.) that focus on removing redundancy in data, generative compression uses generative models to capture the statistical characteristics and structural information of data and represent data in a more efficient form. The basic idea is to first learn the distribution of data through a generative model, and then only store the parameters of the model and a compact representation of each sample in the model's latent space (such as a low-dimensional vector) to reconstruct the original data. This method is suitable for high-dimensional and high-complexity data, such as images, videos, and audio. The advantage of generative compression is that it can achieve high compression rates while maintaining good reconstruction quality, especially for small distortions that are visually acceptable. However, this usually requires the model to accurately learn the rich changes and details of the data, and the compression and decompression process may be more complex and computationally intensive than traditional compression methods.

[0043] ROI (Region Of Interest): is a concept widely used in image processing, computer vision, medical imaging, and machine learning. It refers to a part of an image, video frame, or other form of data that users or algorithms pay special attention to. This area is considered to contain valuable information for a specific task that needs further analysis or processing. In different application contexts, the determination and use of ROI vary, but the core idea is to focus on the most important part of the data to improve processing efficiency or analysis accuracy.

[0044] It should be noted that image compression technology based on deep learning has developed rapidly in recent years. Compared with deep learning compression schemes that use MSE (Mean Squared Error) or MS-SSIM (Multi-Scale Structural Similarity) as loss functions for optimization, generative compression using generative models can reconstruct high-fidelity images from low-bitrate compressed images, so that when compressing images, images can be compressed into images with the lowest bitrate possible, reducing the space occupied by the image. However, using generative models for generative compression cannot guarantee the fidelity of the reconstructed image, resulting in structural and color distortions in the reconstructed image. These distortions are particularly prominent in complex areas such as faces and text. Therefore, generative models are not suitable for reconstructing complex areas such as faces and text at low bitrates.

[0045] In addition, ROI (Region Of Interest) coding is an image and video compression technology based on the region of interest. ROI coding can divide the image into foreground and background, and adopt different coding methods for the foreground and background to save bit rate while ensuring the reconstruction quality of the image and video. For example, a smaller quantization order is used for the foreground area of ​​interest in the image, so as to allocate more bit rate to improve the picture quality, while a larger quantization order is used for the background area of ​​no interest, so as to allocate less bit rate, and the picture quality of this part of the area will be reduced. In this way, without losing the overall image quality, the network bandwidth occupation and video storage space can be saved, or, without increasing the network bandwidth occupation and storage space, the overall quality of the video can be improved.

[0046] The current deep learning-based coding scheme has false detection and missed detection of regions of interest, which makes the division of foreground and background in the image unreasonable. As a result, the key regions of interest will be distorted when the compressed image video is reconstructed. In addition, the higher the bit rate during image compression, the better the quality of the compressed and reconstructed image, but the required storage space and transmission pressure will also be greater. When the current compression model compresses images, it is difficult to balance image quality and compression bit rate, and the image compression effect is poor.

[0047] Therefore, in order to solve the above problems, the embodiments of this specification provide a training method, an image compression method and a device for an image compression model. When training the image compression model, the sample image is divided into a background area and at least one foreground area. Different loss functions are used for the background area and the at least one foreground area to fully utilize the characteristics of different loss functions to constrain the image reconstruction targets of different areas, that is, different loss functions are used to constrain the generation of reconstructed images in different areas, so that the image compression model obtained by training can use different encoding methods and allocate different bit rates for different areas, ensuring better compression effects for different areas. In addition, the background area and at least one foreground area are obtained by performing multiple detections on the sample image, thereby ensuring the accuracy of the division of the foreground area and the background area, so that the trained image compression model can allocate more bit rates to the foreground area of ​​interest when compressing the image, and can better reconstruct and generate the image when decompressing the image, while improving the fidelity of the foreground area of ​​interest in the reconstructed image.

[0048] In recent years, with the rapid development of information technology, the amount of image data has continued to grow. In order to effectively store and transmit these image data, image compression technology has become particularly important. The rise of artificial intelligence (AI) technology has brought new solutions to the field of image compression. Artificial intelligence (AI) technology, especially deep learning, has made significant progress in the field of image compression. Traditional image compression standards such as JPEG, JPEG 2000, WebP, etc. mainly rely on mathematical transformation (such as discrete cosine transform DCT), quantization and entropy coding technologies, while AI-based coding technology can use deep learning, generative adversarial networks (GAN), convolutional neural networks (CNN) and other algorithms to more intelligently identify redundant information in images by learning the intrinsic structure of data, and achieve more efficient and higher-quality image compression.

[0049] AI-based image compression methods mainly include end-to-end trainable image compression models, which use deep learning models to perform analysis (encoding) and synthesis (decoding) processes. They usually include an encoder network for feature extraction and quantization, and a decoder network for image reconstruction. Deep learning models can achieve better compression effects by minimizing the trade-off between reconstruction error and bit rate through training.

[0050] Figure 1 is a schematic diagram of a processing process of an image compression method based on deep learning provided by an embodiment of this specification, such as Figure 1 As shown in the figure, the image compression model (i.e., the deep learning model) includes an encoder and a decoder, obtains the image to be compressed, inputs the image to be compressed into the encoder, uses the encoder to quantize and encode the image to be compressed, obtains quantization features, encodes the quantization features into a bitstream file, and stores the bitstream file. When decoding is required, the compressed bitstream file can be first decoded to obtain the quantization features, and then the quantization features are input into the decoder for decoding to obtain the corresponding reconstructed image.

[0051] The key technical points of the image compression method based on deep learning include: quantization, which is used to reduce redundant information in image representation and is a key step in reducing the bit rate in the compression process; entropy coding, including Huffman coding, arithmetic coding, etc., which is used to further reduce the encoding bit rate; adaptive learning, learning the encoding strategy through training data sets, so that the model can be adaptively adjusted according to the content of the input image.

[0052] AI-based image compression technology has been widely used in many fields such as online video streaming services, mobile communications, satellite imaging, and medical imaging.

[0053] In this specification, a training method for an image compression model is provided. This specification also relates to an image compression method, a training device for an image compression model, an image compression device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0054] See also Figure 2 , Figure 2 A flowchart of a method for training an image compression model provided according to an embodiment of the present specification is shown, which specifically includes the following steps 202-206.

[0055] Step 202: Acquire a sample image, and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image.

[0056] It should be noted that the training method of the image compression model provided in the embodiments of this specification can be applied to the client or to the cloud server. If applied to the client, a large number of sample images can be obtained through the cloud server or other devices to perform model training on the client through the sample images; if applied to the cloud server, a large number of sample images can be obtained from the client, other servers or the cloud server locally to perform model training on the cloud server through the sample images. Among them, the cloud server can be connected to one or more clients through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The clients here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client can interact with the user through a graphical user interface to achieve the upload and management of sample images, as well as the control operation of model training, thereby realizing the training method of the image compression model provided in the embodiments of this specification.

[0057] It should be noted that the sample image is an image obtained for training the model, and the sample image can be multiple unrelated images or a video frame image obtained from a video, which is not limited in the embodiments of this specification. In addition, the region information refers to the identification information of at least two regions obtained by dividing the sample image into regions, such as region coordinates, region labels, etc., which are used to identify the regions obtained by division.

[0058] In actual implementation, models, algorithms or programming techniques can be used to perform region segmentation on the image, such as threshold segmentation, region growing, saliency detection, etc. Specifically, the image can be region segmented based on the foreground and background, or based on the objects included in the image, to obtain at least two regions included in the sample image, and store the corresponding region information.

[0059] In an optional implementation of this embodiment, dividing the sample image to obtain region information includes:

[0060] Using the detection model, dividing the sample image into a target background region and at least one target foreground region, wherein the at least one target foreground region is a non-overlapping image region;

[0061] The region information of the target background region and at least one target foreground region is stored.

[0062] The detection model may refer to a pre-trained model that can identify the foreground and background in a sample image, such as a saliency detection network, an image segmentation network, etc. Among them, the saliency detection network (Salient Object Detection Networks) is a deep learning model in the field of computer vision, which is used to automatically identify and segment the most prominent or visually attractive areas in an image, namely, salient objects. The goal of this type of network is to accurately extract the parts that may be of interest from complex scenes, such as main characters, animals, special events, or visually significant objects, while excluding background or irrelevant details. The saliency detection network is usually built based on deep convolutional neural networks (CNNs).

[0063] It should be noted that the detection model can be used to divide the sample image into a target background area and at least one non-overlapping target foreground area, thereby realizing the division of the background area and the foreground area in the sample image, and the foreground area can be further subdivided into multiple areas, where the foreground area refers to the area in the sample image that can attract visual attention, and the background area refers to the irrelevant area in the sample image.

[0064] In the embodiments of the present specification, at least one foreground area that attracts visual attention and an irrelevant background area in the sample image can be divided, so that in the subsequent model training process, compression control can be performed separately for the foreground area that attracts visual attention and the irrelevant background area, so as to achieve different encoding bit rates for the area that attracts visual attention and the irrelevant background area, thereby ensuring the fidelity of the foreground area that attracts visual attention and improving the compression effect.

[0065] In an optional implementation of this embodiment, the sample image is divided into a target background area and at least one target foreground area using a detection model, including:

[0066] Input the sample image into the detection model to obtain the i-th background area and the i-th foreground area, where i is initially 1;

[0067] Input the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area;

[0068] Let i be incremented by 1, and return to execute the operation steps of inputting the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area, until the detection condition is met, and the n-th background area and the n-th foreground area are obtained;

[0069] The nth background region is taken as the target background region, and the i-th to n-th foreground regions are taken as the target foreground regions.

[0070] Among them, the detection condition is a pre-configured condition used to determine whether the division of the sample image is completed. The detection condition can be configured based on the actual division requirements of different application scenarios. For example, the detection condition can be that the number of divisions reaches a threshold, the foreground area obtained by the current division reaches a size threshold, the foreground area obtained by the current division does not include a detectable object, etc.

[0071] In actual implementation, the sample image can be input into the detection model, and the first detection process is performed to obtain the first background area B_1 and the first foreground area F_1; the first background area B_1 is continuously input into the detection model, and the second detection process is performed to obtain the second background area B_2 and the second foreground area F_2; the second background area B_2 is continuously input into the detection model, and the third detection process is performed to obtain the third background area B_3 and the second foreground area F_3, ...; until the n-1th background area B_n-1 is input into the detection model, and the nth detection process is performed to obtain the nth background area B_n and the nth foreground area F_n. At this time, the nth background area B_n obtained in the nth detection process can be used as the target background area obtained by division, and the first foreground area F_1-nth foreground area F_n obtained in the first detection process-nth detection process is used as the target foreground area obtained by division, that is, the target foreground area is F_{1, ..., n}.

[0072] It should be noted that, in addition to the above-mentioned use of the same detection model to perform multiple detection processes, different detection models can also be used for region detection in actual implementation to obtain a target background region and at least one target foreground region. In specific implementation, it includes n detection models, inputting a sample image into the i-th detection model to obtain the i-th background region and the i-th foreground region, wherein i is initially 1; inputting the i-th background region into the i+1-th detection model to obtain the i+1-th background region and the i+1-th foreground region; letting i be incremented by 1, and returning to execute the operation steps of inputting the i-th background region into the i+1-th detection model to obtain the i+1-th background region and the i+1-th foreground region, until the detection conditions are met, obtaining the n-th background region and the n-th foreground region, taking the n-th background region as the target background region, and taking the i-th to n-th foreground regions as the target foreground regions.

[0073] As an optional example, the detection model may include a face detection network, a vehicle detection network, a text detection network and a building detection network. At this time, the sample image can be input into the face detection network (i.e., the first detection model) to obtain the face area (i.e., the first foreground area) and the first background area; the first background area is continued to be input into the vehicle detection network (i.e., the second detection model) to obtain the vehicle area (i.e., the second foreground area) and the second background area; the second background area is input into the text detection network (i.e., the third detection model) to obtain the text area (i.e., the third foreground area) and the third background area; the third background area is input into the building detection network (i.e., the fourth detection model) to obtain the building area (i.e., the fourth foreground area) and the fourth background area. At this time, the target foreground area includes the face area, the vehicle area, the text area and the building area, and the target background area is the fourth background area.

[0074] In an embodiment of the present specification, when training an image compression model, a sample image can be divided into a background area and at least one foreground area, and the background area and at least one foreground area are obtained by performing multiple detections on the sample image, thereby ensuring the accuracy of the division of the foreground area and the background area, so that the trained image compression model can allocate more bit rate to the foreground area that attracts visual attention when compressing the image, and can better reconstruct and generate the image when decompressing the image, while improving the fidelity of the foreground area that attracts visual attention in the reconstructed image.

[0075] Step 204: Input the sample image and the region information into the initial image compression model to obtain the corresponding predicted reconstructed image.

[0076] The initial image compression model is an initial model that has not been subjected to compression training, and the initial image compression module may include an initial encoder and an initial decoder.

[0077] In actual implementation, the sample image can be input into the initial encoder, which performs feature encoding on the sample image to output image features, and then the image features are compressed and encoded to obtain a compressed code stream. When decoding is required, the compressed code stream can be decoded to obtain the image features, and then the image features can be input into the initial decoder for decoding to obtain the predicted reconstructed image.

[0078] Specifically, the image features output by the initial encoder are compressed and encoded to obtain a compressed code stream. During the training process, the gradient must be returned, so it cannot be stored as a code stream file. Therefore, the code rate of the image features can be calculated to participate in the training of the image compression model. As an example, the image features output by the initial encoder can be encoded in an entropy coding manner, and the entropy of the image features is calculated to participate in the calculation of the subsequent loss value to achieve model training. Among them, entropy coding is a data compression technology that reduces the number of bits required for data representation based on the concept of entropy in information theory. Entropy is a measure of information content or uncertainty. In the context of data compression, the entropy coding method uses the probability distribution of symbols (such as characters, words, pixel values, etc.) in the data to optimize the code length, so that symbols with high frequency use shorter codes, while symbols with low frequency use longer codes, which can reduce the number of bits required to represent the entire data set as a whole.

[0079] In the embodiments of the present specification, after the sample image is divided into regions to obtain region information, the sample image and the region information can be input into the initial image compression model together, and the region information can be used as an additional training information, so that different regions in the sample image can be compressed using different bit rates.

[0080] Step 206: According to the predicted reconstructed image and the sample image, the model reconstruction loss is determined using loss functions corresponding to at least two regions, and the initial image compression model is trained according to the model reconstruction loss to obtain a trained target image compression model, wherein the loss functions corresponding to at least two regions are different.

[0081] Among them, the model reconstruction loss is used to indicate the difference between the predicted reconstructed image and the sample image, that is, the difference between the original input sample image and the reconstructed image predicted by the model, indicating the reconstruction effect of the model.

[0082] It should be noted that the sample image and region information are input into the initial image compression model, and after the initial encoder and the initial decoder, the output predicted reconstructed image can be obtained. The training goal of the initial image compression model is to make the predicted reconstructed image as close as possible to the original input sample image, that is, the output of the target image compression model obtained through training is as close as possible to the input, ensuring the quality of the reconstructed image.

[0083] In actual implementation, the sample image is divided into at least two regions, and the loss functions corresponding to at least two regions are different. Different loss functions are used to constrain the image reconstruction targets of different regions, so as to achieve different compression effects corresponding to different regions. Specifically, the model reconstruction loss can be determined based on the predicted reconstructed image and the sample image using the loss functions corresponding to at least two regions, and then the model parameters of the initial image compression model can be adjusted based on the reverse gradient propagation of the model reconstruction loss, and the operation step of obtaining the sample image is returned until the training stop condition is reached to obtain the trained target image compression model.

[0084] In one possible implementation, it can be determined whether the training stop condition is met based only on the relationship between the model reconstruction loss and the loss threshold. Specifically, if the model reconstruction loss is greater than or equal to the loss threshold, it means that the predicted reconstructed image and each region of the sample image are quite different, and the image reconstruction capability of the initial image compression model is poor. At this time, it can be determined that the training stop condition is not met, the model parameters of the initial image compression model are adjusted, and the operation steps of obtaining the sample image are returned to continue training the initial image compression model until the model reconstruction loss is less than the loss threshold, indicating that each region of the predicted reconstructed image and the sample image is as close as possible, and the image reconstruction capability of the initial image compression model is good. It is determined that the training stop condition is met, the training is stopped, and the trained target image compression model is obtained. Among them, the loss threshold is the critical value of the loss value, and the value of the loss threshold is selected according to the actual situation, and the embodiments of this specification do not impose any restrictions on this.

[0085] In another possible implementation, in addition to comparing the relationship between the model reconstruction loss and the loss threshold, the number of iterations can also be combined to determine whether the training stop condition is reached. Specifically, if the model reconstruction loss is greater than the loss threshold, it can be further determined whether the number of iterations at this moment has reached the preset number of iterations. If the number of iterations at this moment has not reached the preset number of iterations, it can be determined that the training stop condition has not been reached, and the model parameters of the initial image compression model can be adjusted, and the operation steps of obtaining the sample image are returned to continue training the initial image compression model until the preset number of iterations is reached. It is determined that the training stop condition is reached, and the iteration is stopped to obtain the trained target image compression model. Among them, the preset number of iterations is set according to the actual situation. When the number of training times reaches the preset number of iterations, it means that the number of training times of the initial image compression model is sufficient. At this time, the predicted reconstructed image of the initial image compression model is extremely close to the input sample image, and the training can be stopped.

[0086] In the embodiments of the specification, the image reconstruction effect of the initial image compression model can be intuitively shown by calculating the model reconstruction loss, and the initial image compression model is trained based on the model reconstruction loss, and the parameters of the initial image compression model are adjusted, thereby effectively improving the training rate and training effect of the initial image compression model. In addition, different loss functions are used to constrain different areas of the sample image to ensure that different areas of the sample image are optimized, so that the image compression model obtained by training can use different encoding methods and allocate different bit rates for different areas to ensure better compression effects in different areas.

[0087] In an optional implementation of this embodiment, the method further includes:

[0088] Configure corresponding loss functions for at least two regions respectively; or,

[0089] For at least two regions, respectively configure corresponding loss functions and constraint weights;

[0090] Among them, at least two regions correspond to different loss functions and / or different constraint weights.

[0091] In one possible implementation, corresponding loss functions may be configured for at least two regions respectively, that is, a loss function may be configured for each region respectively, so as to apply different constraints to different regions to affect the model reconstruction loss, thereby affecting the fidelity of image reconstruction.

[0092] In another possible implementation, in addition to configuring corresponding loss functions for at least two regions, corresponding constraint weights may also be configured for at least two regions. The constraint weight refers to the degree of constraint of the loss function. The higher the constraint weight, the greater the constraint of the loss function, and the greater the impact on the model reconstruction loss.

[0093] In the embodiments of this specification, the coding control of different regions can be achieved by adjusting the loss functions corresponding to different regions, or the loss functions and constraint weights corresponding to different regions, ensuring different fidelity of different regions, so as to reasonably allocate bit rate resources and improve image compression effect.

[0094] In an optional implementation of this embodiment, corresponding loss functions are configured for at least two regions respectively, including:

[0095] If the at least two regions include a target background region and at least one target foreground region, configuring the target background region and the at least one target foreground region to have different loss functions, and the at least one target foreground region to have the same or different loss functions;

[0096] Among them, the loss function of at least one target foreground area and the loss function of the target background area are used to constrain the image quality of at least one target foreground area to be higher than that of the target background area.

[0097] It should be noted that if the sample image is divided into a target background area and at least one target foreground area, since the foreground area is an area that attracts visual attention, the required reconstruction quality is higher, and thus the impact on the model reconstruction loss is greater. The background area is an irrelevant area in the sample image, and the required reconstruction quality is lower, and thus the impact on the model reconstruction loss is smaller.

[0098] In actual implementation, the loss function of the target background area and at least one target foreground area can be configured to be different, that is, the target background area is configured to correspond to the first loss function, and the target foreground area can be configured with a second loss function. The second loss function constraint and the constraint target of the second loss function are that the image quality of at least one target foreground area is higher than that of the target background area.

[0099] In addition, the importance of different foreground areas in the sample image is also different, that is, the degree to which they attract visual attention is different. Therefore, for at least one target foreground area, in addition to configuring the same loss function, different loss functions can also be configured to achieve different quality constraints for different target foreground areas.

[0100] As an example, a first loss function may be configured for the target background area for training, and a second loss function may be assigned to the target foreground area for training. The first loss function may be a hybrid loss function, and the first loss function also includes at least one of the following: LPIPS (Learned Perceptual Image Patch Similarity, perceptual loss function, also known as learned perceptual image block similarity); mean absolute error loss function (L1 loss, also known as MAE); generative loss function (GAN loss). Among them, LPIPS and L1 loss are both used to measure the difference between two images, which can maximize the reconstruction of background details. LPIPS measures the similarity of two images in a way that is more in line with human judgment. The lower the LPIPS value, the more similar the two images are, and vice versa, the greater the difference. While reducing the loss value of GAN through training, the background areas of the sample image and the predicted reconstructed image gradually meet the two indicators of LPIPS and L1 loss.

[0101] In addition, different constraint weights can be configured for the second loss function corresponding to at least one target foreground area, that is, the second loss functions corresponding to different target foreground areas have different degrees of constraint in the model reconstruction loss, and the greater the constraint weight, the greater the loss function constraint on the target foreground area. Furthermore, the second loss functions corresponding to at least one target foreground area can be the same or different, for example, different second loss functions are used for different target foreground areas to impose different loss function constraints; for another example, when the same second loss function is used, the constraint weights corresponding to the second loss functions corresponding to different target foreground areas are different, and the higher the constraint weight, the greater the impact on the model reconstruction loss.

[0102] For example, assuming that the target foreground area includes the face area, the vehicle area, the text area and the building area, the constraint weight of the second loss function corresponding to the face area can be set to be the highest, followed by the text area, the vehicle area, and the building area, so that when compressing the image, the encoder in the trained target image compression model allocates the highest bit rate to the face area, followed by the text area, the vehicle area, and the building area. In addition, the constraint weight corresponding to the first loss function corresponding to the background area is the lowest, and the bit rate allocated to the background area is also the lowest. When the decoder in the trained target image compression model reconstructs the compressed image, the fidelity of the face area in the foreground area is the highest, and the building area is the worst.

[0103] In the embodiments of the present specification, the constraint weight of the first loss function corresponding to the background area can be configured to be the lowest, and the bit rate allocated to the background area is also the lowest. By adjusting the loss functions and corresponding constraint weights corresponding to the target background area and at least one target foreground area, the image quality of the target foreground area is ensured to be higher than that of the target background area, and the fidelity of the foreground area that attracts visual attention is higher. The bit rate resources are reasonably allocated so that more bit rate can be allocated to the foreground area that attracts visual attention when image compression is performed, so that the image quality of the focus area or other areas is higher and the details are richer.

[0104] In an optional implementation of this embodiment, according to the predicted reconstructed image and the sample image, the model reconstruction loss is determined using loss functions corresponding to at least two regions, including:

[0105] Using the target loss function, the pixel error of each pixel point in the target area in the predicted reconstructed image and the sample image is calculated to obtain the regional pixel loss of the target area, wherein the target area is any one of the at least two areas, and the target loss function is the loss function corresponding to the target area;

[0106] A model reconstruction loss is determined based on the region pixel losses of at least two regions.

[0107] In actual implementation, if corresponding loss functions are configured for at least two regions respectively, any region can be used as the target region, and the pixel error of each pixel point in the target region in the predicted reconstructed image and the sample image is calculated as the regional pixel loss of the region. Based on the regional pixel loss of each region, the model reconstruction loss is determined.

[0108] As an example, it is assumed that the sample image includes foreground area 1, foreground area 2, foreground area 3 and background area, and foreground area 1 corresponds to loss function 1, foreground area 2 corresponds to loss function 2, foreground area 3 corresponds to loss function 3, and background area corresponds to loss function 4. Using the calculation method of loss function 1, the pixel error of each pixel in foreground area 1 in the predicted reconstructed image and the sample image is calculated as the regional pixel loss 1 corresponding to foreground area 1; using the calculation method of loss function 2, the pixel error of each pixel in foreground area 2 in the predicted reconstructed image and the sample image is calculated as the regional pixel loss 2 corresponding to foreground area 2; using the calculation method of loss function 3, the pixel error of each pixel in foreground area 3 in the predicted reconstructed image and the sample image is calculated as the regional pixel loss 3 corresponding to foreground area 3; using the calculation method of loss function 4, the pixel error of each pixel in the background area in the predicted reconstructed image and the sample image is calculated as the regional pixel loss 4 corresponding to the background area. According to regional pixel loss 1, regional pixel loss 2, regional pixel loss 3 and regional pixel loss 4, the overall model reconstruction loss of the initial image compression model is determined.

[0109] It should be noted that corresponding loss functions are used for the predicted reconstructed image and the sample image in different regions, and the corresponding regional pixel losses are calculated. The model reconstruction loss is determined by combining the regional pixel losses of each region. When calculating the model reconstruction loss, different constraints are used for different regions, which in turn affects the fidelity of reconstruction of different regions in the image, achieves different compression qualities in different regions, and improves the image compression effect.

[0110] In an optional implementation of this embodiment, a target loss function is used to calculate the pixel error of each pixel point in the target area in the predicted reconstructed image and the sample image to obtain the regional pixel loss of the target area, including:

[0111] The target loss function is used to calculate the pixel error of each pixel in the target area between the predicted reconstructed image and the sample image;

[0112] The regional pixel loss of the target area is calculated based on the pixel error and the target constraint weight, where the target constraint weight is the constraint weight corresponding to the target loss function.

[0113] It should be noted that in addition to configuring corresponding loss functions for at least two regions respectively, corresponding constraint weights can also be configured for at least two regions respectively. Therefore, the regional pixel loss of at least two regions can be calculated by combining the loss functions corresponding to the at least two regions and the constraint weights corresponding to the loss functions.

[0114] In actual implementation, if corresponding loss functions and corresponding constraint weights are configured for at least two regions respectively, then any region can be used as the target region. The pixel error of each pixel point in the target region in the predicted reconstructed image and the sample image is first calculated based on the corresponding target loss function, and then the pixel error is multiplied by the corresponding constraint weight to obtain the regional pixel loss of the target region.

[0115] In the embodiments of the present specification, corresponding loss functions are used for the predicted reconstructed image and the sample image in different regions, and the pixel error in the corresponding region is calculated and then multiplied by the corresponding constraint weight to obtain the regional pixel loss of the region. The model reconstruction loss can be determined by combining the regional pixel losses of each region. When calculating the model reconstruction loss, different constraints are applied to different regions through loss functions and constraint weights, which in turn affects the regional pixel loss of the sample image, thereby achieving different compression qualities in different regions and improving the image compression effect.

[0116] In an optional implementation of this embodiment, the method further includes:

[0117] Determining encoding bit rates for at least two regions;

[0118] Accordingly, the regional pixel loss of the target area is calculated based on the pixel error and the target constraint weight, including:

[0119] The regional pixel loss of the target area is calculated based on the pixel error, the target constraint weight and the encoding bit rate of the target area.

[0120] In a specific implementation, the image features output by the encoder of the initial image compression model may be obtained, entropy coding may be performed on the image features, and the entropy of each region may be determined as the encoding bit rate of each region.

[0121] Specifically, the regional pixel loss of the i-th region can be determined by the following formula (1):

[0122] L i =R i +λ i D i (1)

[0123] Among them, L i is the regional pixel loss of the ith region; R i is the encoding rate of the i-th region; i is the constraint weight of the loss function corresponding to the i-th region; D i The pixel loss is calculated for each pixel in the ith region using the loss function corresponding to the ith region.

[0124] It should be noted that when calculating the regional pixel loss of each area, in addition to considering different loss functions and constraint weights for different areas, the encoding bit rates of different areas can also be considered. The encoding bit rates of different areas are also used as loss values ​​to train the initial image compression model, so as to allocate different bit rates to different areas for image compression and ensure different compression qualities for different areas.

[0125] In actual implementation, after obtaining the regional pixel losses of at least two regions, the pixel losses of each region can be summed up as the model reconstruction loss of the initial image compression model as a whole, and the initial compression model can be trained based on the model reconstruction loss.

[0126] Of course, in actual implementation, in addition to calculating the regional pixel loss by combining the loss function and constraint weights corresponding to the region and the encoding bit rate of the region as mentioned above, the regional pixel loss can also be calculated by other methods, such as only considering the loss function corresponding to the region to calculate the regional pixel loss; or, combining the loss function and constraint weights corresponding to the region to calculate the regional pixel loss, and the embodiments of this specification do not limit this.

[0127] By applying the solution provided in the embodiments of this specification, the sample image can be divided into at least two regions. Each region uses a corresponding loss function, constraint weight and encoding bit rate to determine the model reconstruction loss, so as to train the initial image compression model, so as to make full use of the characteristics of different loss functions and different constraint weights to constrain the image reconstruction targets of different regions, while considering the encoding bit rates of different regions, so that the target image compression model obtained by training can use different encoding methods and assign different bit rates to different regions, so as to ensure better compression effects in different regions.

[0128] See also Figure 3 , Figure 3 A flowchart of an image compression method provided according to an embodiment of the present specification is shown, which specifically includes the following steps 302-304.

[0129] Step 302: Obtain an image to be compressed.

[0130] Step 304: input the image to be compressed into the target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is trained by the above-mentioned image compression model training method.

[0131] It should be noted that the trained target image compression model includes an encoder and a decoder. The image to be compressed can be input into the encoder, and the encoder performs feature encoding on the image to be compressed to output image features. The image features are then compressed and encoded to obtain a compressed bitstream file, which is then stored. When decoding is required, the compressed bitstream file can be first decoded to obtain the image features, and then the image features can be input into the decoder for decoding to obtain the target reconstructed image.

[0132] Applying the solution provided in the embodiment of this specification, since the target image compression model is Figure 2 The training method of the image compression model shown is obtained by training. The target image compression model can adopt different encoding methods and allocate different bit rates for different areas in the image, thereby ensuring better compression effects in different areas.

[0133] It should be noted that the technical solution of the image compression method and the technical solution of the above-mentioned image compression model training method belong to the same concept. For details not described in detail in the technical solution of the image compression method, please refer to the description of the technical solution of the above-mentioned image compression model training method.

[0134] The following combination Figure 4 , taking the application of the training method of the image compression model provided in this specification in the content interaction scenario as an example, the image compression method is further explained. Figure 4A processing flow chart of an image compression method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0135] Step 402: Acquire a sample interaction image, divide the sample interaction image into a target background area and at least one target foreground area using a detection model, and store area information of the target background area and at least one target foreground area, wherein at least one target foreground area is a non-overlapping image area.

[0136] Step 404: Input the sample image and the region information into the initial image compression model to obtain a corresponding predicted reconstructed image.

[0137] Step 406: For the target background area and at least one target foreground area, respectively configure corresponding loss functions and constraint weights.

[0138] Among them, the loss function of the target background area is different from that of at least one target foreground area, and the loss function of at least one target foreground area is the same or different; the loss function of at least one target foreground area and the loss function of the target background area are used to constrain the image quality of at least one target foreground area to be higher than that of the target background area.

[0139] Step 408: For the target background area and at least one target foreground area, respectively, the corresponding loss functions are used to calculate the pixel error of each pixel point in the corresponding area in the predicted reconstructed image and the sample image; the pixel error is multiplied by the corresponding constraint weight and superimposed with the corresponding encoding bit rate to obtain the regional pixel loss of the area; the regional pixel losses of the target background area and at least one target foreground area are superimposed to obtain the overall model reconstruction loss.

[0140] Step 410: Train the initial image compression model according to the model reconstruction loss, and return to execute the operation of step 402 until the training stop condition is reached to obtain a trained target image compression model.

[0141] It should be noted that the above steps 402-410 are a training process for training the initial image compression model using a large number of sample interactive images. After the training is completed, the target image compression model obtained through training can be used on the content interactive platform to implement image compression.

[0142] Step 412: Acquire the image to be compressed in the content interaction platform, input the image to be compressed into the trained target image compression model, and obtain the target reconstructed image.

[0143] By applying the solution provided in the embodiments of this specification, the sample image can be divided into at least two regions. Each region uses a corresponding loss function, constraint weight and encoding bit rate to determine the model reconstruction loss, so as to train the initial image compression model, so as to make full use of the characteristics of different loss functions and different constraint weights to constrain the image reconstruction targets of different regions, while considering the encoding bit rates of different regions, so that the target image compression model obtained by training can use different encoding methods and assign different bit rates to different regions, so as to ensure better compression effects in different regions.

[0144] Corresponding to the above method embodiment, this specification also provides an embodiment of a training device for an image compression model. Figure 5 FIG. 1 is a schematic diagram showing a structure of a training device for an image compression model provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0145] The division module 502 is configured to obtain a sample image and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image;

[0146] The first obtaining module 504 is configured to input the sample image and the region information into the initial image compression model to obtain the corresponding predicted reconstructed image;

[0147] The training module 506 is configured to determine the model reconstruction loss based on the predicted reconstructed image and the sample image using the loss functions corresponding to at least two regions, train the initial image compression model based on the model reconstruction loss, and obtain a trained target image compression model, wherein the loss functions corresponding to at least two regions are different.

[0148] Optionally, the division module 502 is further configured to:

[0149] Using the detection model, dividing the sample image into a target background region and at least one target foreground region, wherein the at least one target foreground region is a non-overlapping image region;

[0150] The region information of the target background region and at least one target foreground region is stored.

[0151] Optionally, the division module 502 is further configured to:

[0152] Input the sample image into the detection model to obtain the i-th background area and the i-th foreground area, where i is initially 1;

[0153] Input the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area;

[0154] Let i be incremented by 1, and return to execute the operation steps of inputting the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area, until the detection condition is met, and the n-th background area and the n-th foreground area are obtained;

[0155] The nth background region is taken as the target background region, and the i-th to n-th foreground regions are taken as the target foreground regions.

[0156] Optionally, the training module 506 is further configured to:

[0157] Using the target loss function, the pixel error of each pixel point in the target area in the predicted reconstructed image and the sample image is calculated to obtain the regional pixel loss of the target area, wherein the target area is any one of the at least two areas, and the target loss function is the loss function corresponding to the target area;

[0158] A model reconstruction loss is determined based on the region pixel losses of at least two regions.

[0159] Optionally, the training module 506 is further configured to:

[0160] The target loss function is used to calculate the pixel error of each pixel in the target area between the predicted reconstructed image and the sample image;

[0161] The regional pixel loss of the target area is calculated based on the pixel error and the target constraint weight, where the target constraint weight is the constraint weight corresponding to the target loss function.

[0162] Optionally, the training module 506 is further configured to:

[0163] Determining encoding bit rates for at least two regions;

[0164] The regional pixel loss of the target area is calculated based on the pixel error, the target constraint weight and the encoding bit rate of the target area.

[0165] Optionally, the device further includes a configuration module configured to:

[0166] Configure corresponding loss functions for at least two regions respectively; or,

[0167] For at least two regions, respectively configure corresponding loss functions and constraint weights;

[0168] Among them, at least two regions correspond to different loss functions and / or different constraint weights.

[0169] Optionally, the configuration module is further configured to:

[0170] If the at least two regions include a target background region and at least one target foreground region, configuring the target background region and the at least one target foreground region to have different loss functions, and the at least one target foreground region to have the same or different loss functions;

[0171] Among them, the loss function of at least one target foreground area and the loss function of the target background area are used to constrain the image quality of at least one target foreground area to be higher than that of the target background area.

[0172] The embodiments of the present specification provide a training device for an image compression model, including a division module, a first acquisition module, and a training module. Through the interaction of the above three modules, a sample image can be divided into at least two regions. Each region adopts a corresponding loss function, constraint weight, and encoding bit rate to determine the model reconstruction loss, so as to train the initial image compression model, so as to make full use of the characteristics of different loss functions and different constraint weights to constrain the image reconstruction targets of different regions, and at the same time consider the encoding bit rates of different regions, so that the target image compression model obtained by training can adopt different encoding methods and assign different bit rates to different regions, so as to ensure better compression effects in different regions.

[0173] The above is a schematic scheme of a training device for an image compression model of this embodiment. It should be noted that the technical scheme of the training device for the image compression model and the technical scheme of the training method for the image compression model described above belong to the same concept, and the details of the technical scheme of the training device for the image compression model that are not described in detail can be found in the description of the technical scheme of the training method for the image compression model described above.

[0174] Corresponding to the above method embodiment, this specification also provides an image compression device embodiment, Figure 6 FIG. 2 shows a schematic diagram of the structure of an image compression device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0175] An acquisition module 602 is configured to acquire an image to be compressed;

[0176] The second obtaining module 604 is configured to input the image to be compressed into a target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is obtained by training using the above-mentioned image compression model training method.

[0177] The embodiment of the present specification provides an image compression device, including an acquisition module and a second acquisition module. Through the interaction of the three modules, the target image compression model is used to realize image reconstruction. Since the target image compression model is obtained by the above Figure 2The training method of the image compression model shown is obtained by training. The target image compression model can adopt different encoding methods and allocate different bit rates for different areas in the image, thereby ensuring better compression effects in different areas.

[0178] The above is a schematic scheme of an image compression device of this embodiment. It should be noted that the technical scheme of the image compression device and the technical scheme of the above-mentioned image compression model training method and image compression method belong to the same concept, and the details not described in detail in the technical scheme of the image compression device can be referred to the description of the technical scheme of the above-mentioned image compression model training method and image compression method.

[0179] Figure 7 The block diagram of a computing device according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0180] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0181] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0182] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0183] Among them, the processor 720 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the training method of the above-mentioned image compression model or the steps of the image compression method.

[0184] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned image compression model training method or image compression method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned image compression model training method or image compression method.

[0185] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, can implement the training method of the above-mentioned image compression model or the steps of the image compression method.

[0186] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the training method of the image compression model or the image compression method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the training method of the image compression model or the image compression method.

[0187] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned image compression model training method or image compression method.

[0188] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned image compression model training method or image compression method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned image compression model training method or image compression method.

[0189] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] Computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program codes, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROM), random access memories (RAM), electric carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of computer readable media may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0191] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0192] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0193] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A training method for an image compression model, It is characterized in that include: Acquire a sample image, and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image; Inputting the sample image and the region information into an initial image compression model to obtain a corresponding predicted reconstructed image; According to the predicted reconstructed image and the sample image, the model reconstruction loss is determined using the loss functions corresponding to the at least two regions, and the initial image compression model is trained according to the model reconstruction loss to obtain a trained target image compression model, wherein the loss functions corresponding to the at least two regions are different.

2. The method according to claim 1, It is characterized in that The step of dividing the sample image to obtain region information includes: Using the detection model, the sample image is divided into a target background region and at least one target foreground region, wherein the at least one target foreground region is a non-overlapping image region; The region information of the target background region and the at least one target foreground region is stored.

3. The method according to claim 2, It is characterized in that The method of using the detection model to divide the sample image into a target background area and at least one target foreground area includes: Input the sample image into the detection model to obtain the i-th background area and the i-th foreground area, wherein i is initially 1; Input the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area; Let i be incremented by 1, and return to execute the operation step of inputting the i-th background area into the detection model to obtain the i+1-th background area and the i+1-th foreground area, until the detection condition is met, and the n-th background area and the n-th foreground area are obtained; The nth background area is used as the target background area, and the i-th to n-th foreground areas are used as the target foreground areas.

4. The method according to any one of claims 1 to 3, It is characterized in that The step of determining the model reconstruction loss by using the loss functions corresponding to the at least two regions according to the predicted reconstructed image and the sample image comprises: Using a target loss function, a pixel error of each pixel point in a target area between the predicted reconstructed image and the sample image is calculated to obtain a regional pixel loss of the target area, wherein the target area is any one of the at least two areas, and the target loss function is a loss function corresponding to the target area; The model reconstruction loss is determined according to the regional pixel losses of the at least two regions.

5. The method according to claim 4, It is characterized in that The target loss function is used to calculate the pixel error of each pixel point in the target area between the predicted reconstructed image and the sample image to obtain the regional pixel loss of the target area, including: Using the target loss function, calculate the pixel error of each pixel point in the target area between the predicted reconstructed image and the sample image; The regional pixel loss of the target area is calculated according to the pixel error and the target constraint weight, wherein the target constraint weight is the constraint weight corresponding to the target loss function.

6. The method according to claim 5, It is characterized in that The method further comprises: Determining encoding bit rates of the at least two regions; Accordingly, calculating the regional pixel loss of the target area according to the pixel error and the target constraint weight includes: The regional pixel loss of the target area is calculated according to the pixel error, the target constraint weight and the encoding bit rate of the target area.

7. The method according to claim 1, It is characterized in that The method further comprises: configuring corresponding loss functions for the at least two regions respectively; or, respectively configuring corresponding loss functions and constraint weights for the at least two regions; The at least two regions correspond to different loss functions and / or different constraint weights.

8. The method according to claim 7, It is characterized in that The configuring corresponding loss functions for the at least two regions respectively includes: If the at least two regions include a target background region and at least one target foreground region, configuring the target background region and the at least one target foreground region to have different loss functions, and the at least one target foreground region to have the same or different loss functions; The loss function of the at least one target foreground area and the loss function of the target background area are used to constrain the image quality of the at least one target foreground area to be higher than that of the target background area.

9. An image compression method, It is characterized in that include: Obtain the image to be compressed; The image to be compressed is input into the target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is obtained by training using the image compression model training method described in any one of claims 1 to 8.

10. A training device for an image compression model, It is characterized in that include: A division module is configured to obtain a sample image and divide the sample image to obtain region information, wherein the region information is used to indicate at least two regions included in the sample image; A first obtaining module is configured to input the sample image and the region information into an initial image compression model to obtain a corresponding predicted reconstructed image; The training module is configured to determine the model reconstruction loss based on the predicted reconstructed image and the sample image using the loss functions corresponding to the at least two regions, train the initial image compression model based on the model reconstruction loss, and obtain a trained target image compression model, wherein the loss functions corresponding to the at least two regions are different.

11. An image compression device, It is characterized in that include: An acquisition module, configured to acquire an image to be compressed; The second acquisition module is configured to input the image to be compressed into the target image compression model to obtain a corresponding target reconstructed image, wherein the target image compression model is obtained by training using the image compression model training method described in any one of claims 1 to 8.

12. A computing device, It is characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the training method of the image compression model described in any one of claims 1 to 8 or the steps of the image compression method described in claim 9 are implemented.

13. A computer-readable storage medium, It is characterized in that It stores computer executable instructions, which, when executed by a processor, implement the training method of the image compression model described in any one of claims 1 to 8 or the steps of the image compression method described in claim 9.

14. A computer program product, It is characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the training method of the image compression model described in any one of claims 1 to 8 or the steps of the image compression method described in claim 9.

Images