Image processing method, device, storage medium and electronic device

Abstract

The embodiments of the present disclosure disclose an image processing method, device, storage medium and electronic device. The method includes obtaining an original image, extracting high-frequency information and low-frequency information in the original image; extracting content-related information in the high-frequency information, writing the content-related information into the low-frequency information, and obtaining a low-resolution image corresponding to the original image. Through the technical solution of the embodiments of the present disclosure, the obtained low-resolution image preserves the content-related information in the original image, thereby achieving the goal of reducing the amount of image data while improving image quality.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of computer data processing, and in particular to an image processing method, device, storage medium, and electronic device. Background Art

[0002] Digital images often need to be resampled during storage, display and transmission. On the one hand, the resampled images can be adapted to devices with different resolutions for better browsing. On the other hand, the images are downsampled and stored and transmitted, and then upsampled when displayed on the terminal device. The prior art uses the same set of models for downsampling and upsampling. In the implementation process, downsampling will reduce the image resolution, thereby reducing storage and transmission costs. However, the above-mentioned downsampling will also lead to the loss of high-frequency information in the image, so that the details of the reconstructed image obtained from the low-resolution image are lost during the subsequent upsampling process, thereby reducing the image details of the high-resolution image obtained after the low-resolution image is upsampled, thereby reducing the image quality. Summary of the invention

[0003] The embodiments of the present disclosure provide an image processing method, device, storage medium and electronic device, so as to achieve that the obtained low-resolution image preserves the content-related information in the original image, thereby improving the image quality while reducing the image data volume.

[0004] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising: acquiring an original image, extracting high-frequency information and low-frequency information in the original image;

[0005] Content-related information is extracted from the high-frequency information, and the content-related information is written into the low-frequency information to obtain a low-resolution image corresponding to the original image.

[0006] In a second aspect, an embodiment of the present disclosure further provides another image processing method, the method comprising: acquiring a low-resolution image, and extracting content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0007] The high-frequency information is determined based on the content-related information, and an original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.

[0008] In a third aspect, an embodiment of the present disclosure further provides an image processing device, the device comprising: an information extraction module, configured to obtain an original image and extract high-frequency information and low-frequency information from the original image;

[0009] The low-resolution image generation module is used to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.

[0010] In a fourth aspect, an embodiment of the present disclosure further provides another image processing device, the device comprising: an information extraction module, configured to obtain a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0011] The original image generation module is used to determine the high-frequency information based on the content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

[0012] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] a storage device for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of the embodiments of the present disclosure.

[0016] In a sixth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the image processing method as described in any one of the embodiments of the present disclosure.

[0017] The technical solution of the disclosed embodiment obtains the original image, extracts the high-frequency information and low-frequency information in the original image; extracts the content-related information in the high-frequency information, writes the content-related information into the low-frequency information, and obtains the low-resolution image corresponding to the original image. The above technical solution obtains the high-frequency information of the original image by extracting information from the original image, and further decomposes the high-frequency information to extract the content-related information about the original image from the high-frequency information, and writes the content-related information as stego-information into the down-sampled low-resolution image, thereby achieving a high-resolution image with more texture details in the process of reverse up-sampling the low-resolution image, thereby improving the image quality while reducing the amount of image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0020] Figure 2 is a schematic diagram of a spatial rearrangement process provided by an embodiment of the present disclosure;

[0021] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0022] Figure 4 is a flowchart of another image processing method provided by an embodiment of the present disclosure;

[0023] Figure 5 is a structural schematic diagram of an image processing device provided by an embodiment of the present disclosure;

[0024] Figure 6 is a structural schematic diagram of another image processing device provided by an embodiment of the present disclosure;

[0025] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0029] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0031] In some embodiments, before the image is transmitted or stored, the original image is downsampled to obtain a low-resolution image corresponding to the original image, thereby reducing the amount of original data. The low-resolution image is then transmitted or stored to reduce the data resources occupied during the transmission or storage process, thereby reducing the transmission or storage cost. Further, when the original image is displayed, a low-resolution image of the original image is obtained, and the low-resolution is upsampled to obtain a high-resolution image corresponding to the low-resolution image to present a clearer image. In order to enable the upsampled high-resolution image to restore the image details in the original image as much as possible, the method used in the prior art to downsample the original image in real time includes: mapping the high-frequency information in the original image to a Gaussian distribution, so that the image information on the Gaussian distribution can be extracted in the subsequent upsampling process to obtain a high-resolution image with more original image information. The technical solution of the embodiment of the present disclosure was found to have many limitations during the implementation of the above method; specifically, after obtaining the high-frequency information of the original image, the high-frequency information is not further separated according to the image content. Since all high-frequency information cannot be converted to an easily measurable Gaussian distribution in a lossless and reversible manner, the content-related information carried in the high-frequency information may be lost, and the ability of the downsampled low-resolution image to store a large amount of information invisibly is ignored, resulting in too little image content information being available when the image is upsampled, and insufficient details in the restored original image. In response to the above technical problems, in order to restore more details of the original image in the upsampled high-resolution image, the technical solution provided by the embodiment of the present disclosure, see Figure 1 , Figure 1This is a flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is suitable for downsampling and upsampling an image. The method can be executed by an image processing device provided by an embodiment of the present disclosure. The image processing device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal or a PC. Figure 1 , the method of this embodiment includes:

[0032] S110: Acquire an original image, and extract high-frequency information and low-frequency information from the original image.

[0033] S120: Extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.

[0034] In this embodiment, the high-frequency information and low-frequency information of the original image to be processed are decomposed, and after the high-frequency information of the original image is obtained, the high-frequency information is decomposed to extract content-related information about the original image from the high-frequency information, and the content-related information is written as stego information into the down-sampled low-frequency information to obtain a low-resolution image of the original image, so that the low-resolution image includes more texture information. Accordingly, high-frequency information related to the image content is obtained in the process of inverse upsampling, and a high-resolution image with more texture details is accurately obtained, thereby improving the image quality.

[0035] In the embodiments of the present disclosure, the original image may be understood as an image that has not been processed. The low-resolution image may be understood as an image that contains the image content in the original image and has a reduced data volume.

[0036] Specifically, in order to obtain a low-resolution image of the original image, it is first necessary to extract image information from the original image, and respectively extract high-frequency information and low-frequency information from the original image, wherein the low-frequency information includes image content information, and the high-frequency information includes texture information such as edges.

[0037] In some embodiments, a method for extracting high-frequency information and low-frequency information from an original image may include: down-sampling the original image to obtain low-frequency information; determining initial high-frequency information based on a high-resolution low-frequency image obtained by upsampling the original image and the low-frequency information, and rearranging spatial pixels in the initial high-frequency information to a channel dimension to obtain high-frequency information that matches the low-frequency information.

[0038] In the embodiment of the present disclosure, the original image can be represented as (H, W, C); wherein H represents the height data of the original image, W represents the width data of the original image, and C represents the channel data of the original data. For example, C can be 3, indicating that the channel data of the original image is 3 channels including RGB channels respectively. Specifically, the original image is downsampled to obtain the low-frequency information of the original image, and the obtained low-frequency information is directly upsampled to obtain a high-resolution low-frequency image corresponding to the low-frequency information, and the initial high-frequency information in the original image is determined based on the image data of the original image and the image data of the high-resolution low-frequency image. Exemplarily, the initial high-frequency information in the original image can be determined based on the image data difference between the above two images. For example, the original image and the high-resolution low-frequency image have the same resolution, and the pixel values ​​of corresponding pixels in the original image and the high-resolution low-frequency image are respectively differenced to obtain the initial high-frequency information. Furthermore, the spatial pixels in the initial high-frequency information are rearranged, and the spatial pixels are arranged to the channel dimension, so as to obtain high-frequency information matching the low-frequency information, wherein the resolution of the high-frequency information is the same as that of the low-frequency information. Optionally, such as Figure 2 As shown, rearranging the spatial pixels in the initial high-frequency information to the channel dimension can be understood as reducing the height data and width data in the initial high-frequency information, and arranging the pixel data corresponding to the excess height data and width data in other channels, thereby increasing the number of channel dimensions to obtain high-frequency information that matches the resolution of the low-frequency information. Exemplarily, the low-frequency information can be represented as (H / 2, W / 2, C), and the high-frequency information can be represented as H / 2, W / 2, 4C)

[0039] It should be noted that the above-mentioned matching can be understood as matching the image resolution of the low-frequency information with the high-frequency information, that is, the height data in the low-frequency information matches the height data in the high-frequency information, and the width data in the low-frequency information matches the width data in the high-frequency information. The technical solution of the disclosed embodiment obtains high-frequency information that matches the low-frequency information, and the beneficial effect is that it is convenient to subsequently steganographically write the content-related information extracted from the high-frequency information into the low-frequency information. Among them, steganography can be understood as hiding and storing the content-related information in the original image in the low-frequency information, forming a low-resolution image containing the high-frequency information related to the image content of the original image, and making full use of the ability of the low-resolution image to stealthily store a large amount of information, so that the low-resolution image is subsequently upsampled to obtain a high-resolution image, and part of the information related to the image content in the high-frequency information of the original image is reversely extracted in the process, so as to improve the image quality of the image at the reverse restoration.

[0040] Exemplarily, the original image with a resolution of (H, W, C) is downsampled by 2 times, that is, bicubic interpolation is used to downsample by 2 times to obtain low-frequency information (H / 2, W / 2, C) of the original image; further, the low-frequency information (H / 2, W / 2, C) is upsampled by 2 times using bicubic interpolation to obtain a high-resolution low-frequency image (H, W, C) with the same width data and length data as the original image, and the above high-resolution low-frequency image is subtracted from the image data of the original image to obtain the initial high-frequency information of the original image; further, the spatial pixels in the initial high-frequency information are rearranged to arrange the spatial pixels to the channel dimension, so as to obtain high-frequency information (H / 2, W / 2, 4C) matching the low-frequency information.

[0041] In some embodiments, the method for extracting high-frequency information and low-frequency information in the original image may further include: performing a filtering transformation on the original image to obtain the high-frequency information and low-frequency information in the original image. Among them, the filtering transformation may adopt a wavelet transform, and specifically, the Haar transform may be used to perform high-pass filtering on the input information to obtain separated high-frequency information and low-frequency information. Specifically, the image information in the original image is subjected to high-pass filtering using a filtering transformation to obtain the high-frequency information and low-frequency information of the original image. Exemplarily, using a filtering transformation, the original image (H, W, C) is subjected to information separation by high-pass filtering to obtain low-frequency information (H / 4, W / 4, C) and high-frequency information (H / 4, W / 4, 15C) after being downsampled 4 times, respectively.

[0042] In the above downsampling process, the downsampling multiples of the original image include 2 and 4. It should be noted that the above sampling multiples are only introduced as an optional embodiment for exemplary purposes, and the technical solution of this embodiment can also adopt other sampling multiples, which are not limited in this embodiment.

[0043] After obtaining the high-frequency information and low-frequency information of the original image, the high-frequency information of the original image is further extracted to obtain content-related information in the high-frequency information, and the content-related information is written into the low-frequency information, thereby obtaining a low-resolution image corresponding to the original image.

[0044] It should be explained that the high-frequency information of the original image includes content-related information and content-irrelevant information. Content-related information can be understood as information related to the image content in the original image; content-irrelevant information can be understood as other information such as image noise in the original image. Specifically, the method for extracting content-related information in the high-frequency information can be extracted using a pre-trained neural network model or other method. Optionally, the high-frequency information can be processed based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information. Specifically, the information extraction model can be a network structure module, and the network structure of the information extraction model can be a structure such as a convolutional neural network, a multi-layer perceptron, etc., which is not limited to this.

[0045] On the basis of the above embodiments, the information extraction model may be an attention reversible transformation network model. Accordingly, a method for processing high-frequency information based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information may include: inputting high-frequency information and low-frequency information into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, wherein the low-frequency information is an auxiliary condition for extracting content-related information from the high-frequency information.

[0046] It should be noted that the low-frequency information contains the content information of the original image, that is, the low-frequency information is used as an auxiliary condition for extracting content-related information and is input into the attention reversible transformation network model at the same time as the high-frequency information. The content information contained in the low-frequency information can be used as the learning target of the attention reversible transformation network when extracting information, so that the attention reversible transformation network can accurately extract the image content information of the original image in the high-frequency information when extracting information.

[0047] Before using the attention reversible transformation network model for information extraction, it is necessary to first train the attention reversible transformation network model. Specifically, the attention reversible transformation network model training method may include: inputting the high-frequency information in the original sample image into the attention reversible transformation network model to be trained, obtaining content-related information and content-irrelevant information output by the attention reversible transformation network model, and reversely inputting the content-related information and content-related information into the attention reversible transformation network model to be trained, and obtaining the predicted high-frequency information output by the attention reversible transformation network model in reverse. Based on the predicted high-frequency information and the high-frequency information in the original image, a loss function of the attention reversible transformation network model to be trained is generated, and based on the loss function, the model parameters of the attention reversible transformation network model to be trained are adjusted, and the above training process is iteratively performed until a trained attention reversible transformation network model is obtained.

[0048] Exemplarily, the high-frequency information (H / 2, W / 2, 4C) obtained by separating the above information and matching the low-frequency information is further subjected to information extraction. Optionally, based on the attention reversible transformation network model, the high-frequency information (H / 2, W / 2, 4C) is separated into content-related information (H / 2, W / 2, 4pC) and content-irrelevant information (H / 2, W / 2, 4C-4pC) at a separation ratio p. It should be noted that the above is the information separation of high-frequency information downsampled by 2 times; the separation of high-frequency information downsampled by 4 times (H / 4, W / 4, 15C) is content-related information (H / 4, W / 4, 15pC) and content-irrelevant information (H / 4, W / 4, 15C-15pC). Among them, the separation ratio p is pre-set in the attention reversible transformation network model and is not limited to this. Optionally, the separation ratio p corresponding to downsampling by 2 times is greater than the separation ratio p corresponding to downsampling by 4 times. It should be noted that different downsampling multiples can correspond to different attention reversible transformation network models.

[0049] In the implementation of the present disclosure, after respectively obtaining the content-related information of the low-frequency information in the original image and the high-frequency information in the original image, the content-related information is further written into the low-frequency information, thereby obtaining a low-resolution image corresponding to the original image. Optionally, the method of writing the content-related information into the low-frequency information to obtain the low-resolution image corresponding to the original image may include: performing data fusion on the content-related information and the low-frequency information in the channel dimension to obtain the low-resolution image corresponding to the original image.

[0050] Specifically, in order to obtain high-frequency information that matches the low-frequency information, the spatial pixels of the initial high-frequency information in the original image are rearranged to the channel dimension, so that the subsequently obtained high-frequency information matches the height data and width data in the low-frequency information. Accordingly, the channel dimension of the high-frequency information is greater than the channel dimension of the low-frequency information. Accordingly, the channel dimension of the content-related information extracted from the high-frequency information is also greater than the channel dimension of the low-frequency information.

[0051] On this basis, data fusing the content-related information and the low-frequency information in the channel dimension may include: determining each first channel data in the low-frequency information and at least one second channel data in the corresponding content-related information; and data fusing the first channel data and the second channel data having a corresponding relationship.

[0052] In this embodiment, the channel data can be understood as the pixel data in each channel. Correspondingly, the low-frequency information includes three first channels of RGB, and the first channel data can be understood as the R channel data, G channel data and B channel data in the low-frequency information; the second channel data can be understood as the pixel data of each channel in the content-related information; it is worth noting that since the channel dimension of the content-related information is greater than the channel dimension of the low-frequency information, correspondingly, the channel dimension of the first channel data is less than the channel dimension of the second channel data. Therefore, the channel correspondence between the channel data of the first channel and the channel data of the second channel is a one-to-many relationship, that is, it is necessary to determine each first channel data in the low-frequency information and the corresponding at least one second channel data in the content-related information.

[0053] In this embodiment, the correspondence between the first channel data and the second channel data can be preset, and the setting method of the correspondence is not limited. Optionally, the first channel data and the same type of channels of the second channel data have a correspondence relationship. For example, the first channel data includes three RGB channels, and the second channel data includes nine channels, that is, the number of RGB channels is three, respectively. Correspondingly, the R channel in the first channel data corresponds to the three R channels in the second channel data, the G channel in the first channel data corresponds to the three G channels in the second channel data, and the B channel in the first channel data corresponds to the three B channels in the second channel data. Optionally, each channel of the first channel data corresponds to each channel in the second channel data in turn at a polling interval. For example, the resolution of the low-frequency information is (H / 2, W / 2, C), and the channel dimension of the first channel data can be understood as 3 channels; the content-related information is (H / 2, W / 2, 4pC), and taking the preset separation ratio p as 75% as an example, the channel dimension of the second channel data can be understood as 9 channels. Determine the correspondence between the 3 channels in the first channel data and the 9 channels in the second channel data. The corresponding relationship may be that the first channel in the first channel data corresponds to the first channel, the fourth channel, and the seventh channel in the second channel data, the second channel in the first channel data corresponds to the second channel, the fifth channel, and the eighth channel in the second channel data, and the third channel in the first channel data corresponds to the third channel, the sixth channel, and the ninth channel in the second channel data. Optionally, each channel of the first channel data corresponds to each channel in the second channel data in groups in sequence, the resolution of the low-frequency information is (H / 2, W / 2, C), and the content-related information is (H / 2, W / 2, 4pC), and the corresponding relationship may also be that the first channel in the first channel data corresponds to the first channel to the third channel in the second channel data, the second channel in the first channel data corresponds to the fourth channel to the sixth channel in the second channel data, and the third channel in the first channel data corresponds to the seventh channel to the ninth channel in the second channel data. Of course, there may be other corresponding relationships, which are not limited in this embodiment.

[0054] Further, after determining the channel correspondence between the first channel data and the second channel data, the channel data in the first channel data having the corresponding relationship and the channel data in the second channel data are fused to obtain a low-resolution image corresponding to the original image. Specifically, if the downsampling is 2 times, the process of obtaining the low-resolution image may include: ((H / 2, W / 2, C), (H / 2, W / 2, 4pC))-> (H / 2, W / 2, C); if the downsampling is 4 times, the process of obtaining the low-resolution image may include: ((H / 4, W / 4, C), (H / 2, W / 2, 15pC))-> (H / 4, W / 4, C).

[0055] On the basis of the above-mentioned embodiment, after extracting the content-irrelevant information of the high-frequency information, the technical solution of this embodiment further maps the content-irrelevant information to a standard normal distribution. Among them, the standard normal distribution can be understood as a normal distribution with a mean of 0 and a variance of 1. The effect of mapping the content-irrelevant information to the normal distribution is that in the process of upsampling the low resolution to the original image, the normal distribution can be used for information extraction, and the extracted information can be used as the image noise information of the original image for image processing, so as to restore a high-resolution image that is closer to the original image. Of course, in some embodiments, the content-irrelevant information may not be processed. Accordingly, in the process of upsampling the low resolution to restore the original image, information extraction can be directly performed based on the standard normal distribution, and the extracted noise data and the low-resolution image can be processed to obtain a high-resolution image.

[0056] The technical solution of the disclosed embodiment obtains the original image, extracts the high-frequency information and low-frequency information in the original image; extracts the content-related information in the high-frequency information, writes the content-related information into the low-frequency information, and obtains a low-resolution image corresponding to the original image. The above technical solution obtains the high-frequency information of the original image by extracting information from the original image, and further decomposes the high-frequency information to extract the content-related information about the original image from the high-frequency information, and writes the content-related information as stego-information into the down-sampled low-resolution image, thereby achieving a high-resolution image with more texture details in the process of reverse up-sampling the low-resolution image, and improving the image quality while reducing the amount of image data.

[0057] Figure 3 This is a flow chart of another image processing method provided by an embodiment of the present disclosure. The method can be executed by an image processing device provided by an embodiment of the present disclosure. The image processing device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal or a PC. Figure 3, the method of this embodiment includes:

[0058] S210 , obtaining a low-resolution image, and extracting content-related information from low-frequency information and high-frequency information based on the low-resolution image.

[0059] S220 , determining high-frequency information based on content-related information, and obtaining an original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

[0060] In the embodiment of the present disclosure, in order to obtain an original image corresponding to a low resolution, it is necessary to perform upsampling processing on the acquired low resolution image to obtain an original image with a high resolution.

[0061] Specifically, in order to obtain the original image corresponding to the low-resolution image, it is first necessary to extract information from the low-resolution image, and extract the content-related information from the low-frequency information and high-frequency information in the low-resolution image. The content-related information includes part of the high-frequency information of the low-resolution image that is hidden in the process of downsampling the original image to obtain the low-resolution image. It should be noted that the process of extracting low-frequency information and content-related information from the low-resolution image is a reversible process with the process of hiding the content-related information in the low-frequency information in the above-mentioned embodiment, and there is a corresponding relationship between the two processing methods.

[0062] In some embodiments, a method for extracting content-related information in low-frequency information and high-frequency information based on a low-resolution image may include: determining the correspondence between each data channel in the low-frequency information and each data channel in the low-resolution image, and determining each first channel data of the low-frequency information based on each channel data in the low-resolution image; determining the correspondence between each data channel in the content-related information and each data channel in the low-resolution image, and determining each second channel data of the content-related information based on each channel data in the low-resolution image.

[0063] Specifically, the low-resolution image is obtained by data fusion in the channel dimension based on low-frequency information and content-related information, which can be specifically understood as being obtained by data fusion based on channel data of low-frequency information and channel data of content-related information having a corresponding relationship. Accordingly, in the process of extracting low-frequency information and content-related information in a low-resolution image, it is necessary to determine the correspondence between each data channel in the low-frequency information in the low-resolution image and each data channel in the low-resolution image, as well as the correspondence between each data channel in the content-related information and each data channel in the low-resolution image. Determine each first channel data of the low-frequency information based on each channel data in the low-resolution image, and determine each second channel data of the content-related information based on each channel data in the low-resolution image.

[0064] It is worth noting that in the technical solution of the embodiment of the present disclosure, the order of determining the first channel data and the order of determining the second channel data are not prioritized, and can be executed sequentially or simultaneously, and this embodiment does not limit this.

[0065] Exemplarily, if the resolution of the low-resolution image is (H / 2, W / 2, C), then when C=3, it can be understood that the channel dimension of the low-resolution image is 3 channels. Optionally, the correspondence between each channel in the low-frequency information and each data channel in the low-resolution image is obtained. If the correspondence between the data channels is a one-to-one correspondence, the channel dimension of the low-frequency information in the corresponding low-resolution image is also 3 channels, and the resolution of the low-frequency information is (H / 2, W / 2, C); based on the data correspondence of each channel in the low-resolution image, the first channel data (H / 2, W / 2, C) of each channel in the low-frequency information is determined.

[0066] Specifically, the method for determining the first channel data may be to copy the data of each channel in the low-resolution image as the first channel data of the corresponding data channel in the low-frequency information.

[0067] Specifically, the correspondence between each channel in the content-related information and each data channel in the low-resolution image is obtained. Optionally, the correspondence may be that the first channel, the fourth channel and the seventh channel in the content-related information correspond to the first channel in the low-resolution image; based on the channel data of the first channel in the low-resolution image, the channel data of the first channel, the fourth channel and the seventh channel in the second channel data are respectively determined; and the channel data of other channels in the second channel data are correspondingly determined based on the channel correspondence between the remaining channels. Optionally, the correspondence may also be that the first channel to the third channel in the content-related information correspond to the first channel in the low-resolution image; based on the channel data of the first channel in the low-resolution image, the channel data of the first channel to the third channel in the second channel data are respectively determined, and the channel data of other channels in the second channel data are correspondingly determined based on the channel correspondence between the remaining channels. Of course, the correspondence between each data channel in the content-related information and each data channel in the low-resolution image may also be other correspondences, and accordingly, each second channel data of the content-related information is determined based on other correspondences and each channel data in the low-resolution image, which is not limited in this embodiment.

[0068] Based on the above embodiment, the method for determining each second channel data of content-related information based on each channel data in the low-resolution image may include: copying each channel data in the low-resolution image as the second channel data of the corresponding data channel in the content-related information.

[0069] In some embodiments, the method for determining each second channel data of content-related information based on each channel data in the low-resolution image may also include: taking the random value of each channel data in the low-resolution image within a preset range as the second channel data of the corresponding data channel in the content-related information. Exemplarily, the channel data of the first channel to the third channel in the second channel data are determined based on the channel data of the first channel in the low-resolution image as an example for introduction: any data in the first channel in the low-resolution image may be 230, and the preset range of the data may be 230±5. Correspondingly, the corresponding data in the first channel to the third channel in the second channel data may be obtained by random sampling within the range of 230±5. For example, the corresponding data in the first channel in the second channel data may be 232, the corresponding data in the second channel may be 235, and the corresponding data in the third channel may be 228. Further, the second channel data of the corresponding data channel in the content-related information may be expressed as (H / 2, W / 2, 4pC).

[0070] In this embodiment, in order to determine the high-frequency information corresponding to the low-resolution image based on the content-related information, and in order to improve the authenticity of the restored high-resolution data, it is necessary to obtain content-irrelevant data, and determine the high-frequency information based on the content-related data and the content-irrelevant data. Optionally, the method for determining content-irrelevant data may include: resampling the information in a preset data distribution corresponding to the content-irrelevant information to obtain the content-irrelevant information. The preset data distribution can be understood as the normal distribution to which the content-irrelevant information of the high-frequency information in the original image is mapped in the process of downsampling the original data to obtain the low-resolution image; of course, other preset data distributions can also be selected, and this embodiment does not limit this.

[0071] Specifically, the mapped normal distribution is subjected to content-independent information extraction, and the extracted information is used as image noise information of the original image, and information is resampled with the content-related image to obtain high-frequency information corresponding to the low resolution. Of course, in some embodiments, information extraction can also be performed directly based on the standard normal distribution, and content-independent information and content-related information are resampled to obtain high-frequency information.

[0072] In some embodiments, a method for resampling content-irrelevant information and content-related information to obtain high-frequency information may include: reversely inputting content-related information and content-irrelevant information into an attention reversible transformation network model to obtain high-frequency information output by the attention reversible transformation network model.

[0073] Exemplarily, content-related information is extracted from a low-resolution image. If the image is upsampled by 2 times, the content-related information is expressed as (H / 2, W / 2, 4pC); if the image is upsampled by 4 times, the content-related information is expressed as (H / 2, W / 2, 15pC). Furthermore, if the image is upsampled by 2 times, the content-irrelevant information is expressed as (H / 2, W / 2, 4C-4pC); if the image is upsampled by 4 times, the content-irrelevant information is expressed as (H / 4, W / 4, 15C-15pC). Furthermore, if the sampling is downsampled by 2 times, the process of obtaining high-frequency information may include ((H / 2, W / 2, 4C-4pC), (H / 2, W / 2, 4pC))->(H / 2, W / 2, 4C); if the sampling is downsampled by 4 times, the process of obtaining a low-resolution image may include: ((H / 2, W / 2, 15pC), (H / 2, W / 2, 15pC))->(H / 4, W / 4, 15C)).

[0074] Furthermore, after obtaining the low-frequency information and high-frequency information of the low-resolution image, the original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information. Optionally, the method for obtaining the original image corresponding to the low-resolution image may include: upsampling the low-frequency information to obtain a high-resolution low-frequency image; spatially inversely rearranging the channel data of the high-frequency information to obtain a high-resolution high-frequency image, and obtaining the original image based on the high-resolution low-frequency image and the high-resolution high-frequency image.

[0075] Specifically, the obtained low-frequency information is directly upsampled to obtain a high-resolution low-frequency image corresponding to the low-frequency information; and the data channels in the high-frequency information are spatially inversely rearranged to obtain a high-resolution high-frequency image corresponding to the high-frequency information. Furthermore, the high-resolution low-frequency image and the high-resolution high-frequency image are fused to obtain the original image corresponding to the low-resolution image.

[0076] Exemplarily, the original image with a resolution of (H / 2, W / 2, C) is upsampled by a factor of 2, that is, bicubic interpolation is used to upsample it by a factor of 2 to obtain a high-resolution low-frequency image (H, W, C); the high-frequency information (H / 2, W / 2, 4C) is spatially inversely rearranged to obtain a high-resolution high-frequency image (H, W, C) having the same width data and length data as the high-resolution low-frequency image; then the high-resolution low-frequency image and the high-resolution high-frequency image are fused to obtain the original image (H, W, C) corresponding to the low-resolution image.

[0077] In some embodiments, the method for obtaining the original image corresponding to the low-resolution image may further include: performing Haar inverse transform on the high-frequency information and the low-frequency information to obtain the original image.

[0078] Exemplarily, the Haar inverse transform is used to fuse the low-frequency information (H / 4, W / 4, C) and the high-frequency information (H / 4, W / 4, 15C) to obtain the original image (H, W, C) corresponding to the low-resolution image.

[0079] The technical solution of the embodiment of the present disclosure obtains a low-resolution image, extracts content-related information from low-frequency information and high-frequency information based on the low-resolution image, determines high-frequency information based on content-related information, and obtains the original image corresponding to the low-resolution image based on the fusion of high-frequency information and low-frequency information. Through the above technical solution, a high-resolution image with more texture details is obtained, thereby improving image quality.

[0080] On the basis of the above embodiments, the disclosed embodiment further provides an application embodiment for explaining the steps of obtaining a low-resolution image based on a high-resolution image, and then obtaining a high-resolution image based on the low-resolution image.

[0081] Before introducing the following application embodiments, the application scenario of the embodiment is first adaptively introduced. Specifically, the interactive embodiment can be applicable to the process of image transmission. Exemplarily, it can be data transmission between the client and the server. For example, the client requests the server to send an image / video. Accordingly, before the server transmits the image / video, it first processes the image to be transmitted or each video frame in the video to be transmitted (i.e., the high-resolution image) based on the above-mentioned embodiment to obtain the corresponding low-resolution image / video, and transmits the low-resolution image / video to reduce the amount of transmission data, thereby reducing the transmission cost. After receiving the low-resolution image / video sent by the server, the client obtains the high-resolution image / video corresponding to the low-resolution image / video based on the processing method provided in the above-mentioned embodiment for each image frame in the low-resolution image or low-resolution video for image display. Of course, the above-mentioned image / video transmission can also be image transmission between the client and the client, which is not limited in this embodiment. Optionally, the application embodiment can also be used in the process of image / video storage. Exemplarily, during the process of local storage of images by the client, the image frames in the image to be stored or the video to be stored can be processed based on the above-mentioned embodiment to obtain a low-resolution image / video, and the low-resolution image / video can be stored. When the stored image / video needs to be processed or displayed, the image frames in the low-resolution image or the low-resolution video can be processed based on the above-mentioned embodiment to obtain a corresponding high-resolution image / video for subsequent display or processing.

[0082] The above application scenarios are only optional application scenarios of this application embodiment. This embodiment can also be applied to other application scenarios, which will not be described in detail in this embodiment.

[0083] like Figure 4 As shown, the application embodiment specifically includes the following steps:

[0084] Step 1: Separate the high-frequency and low-frequency information of the original image to obtain high-frequency information and low-frequency information respectively.

[0085] Step 2: Separate the high-frequency information to obtain content-related information and content-irrelevant information.

[0086] Step 3: Write content-related information into the low-frequency information to obtain a low-resolution image corresponding to the high-resolution image.

[0087] Step 4: Map content-irrelevant information into Gaussian noise.

[0088] Step 5: Extract information from the low-resolution image to obtain content-related information in the low-frequency information and high-frequency information respectively.

[0089] Step 6: Resample the Gaussian noise to obtain content-irrelevant information.

[0090] Step 7: Fuse content-irrelevant information and content-relevant information to obtain high-frequency information.

[0091] Step 8: Fuse the high-frequency information with the low-frequency information to obtain a high-resolution image.

[0092] It is worth noting that, among the above steps, steps 1 to 4 can be executed by the same device, such as a server, while steps 5 to 8 can be executed by another device, such as a client; steps 1 to 8 can also be executed by the same device, such as a server or a client. This embodiment does not limit the execution device of each step.

[0093] Figure 5 Schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Figure 5 As shown, the device includes: an information extraction module 310 and a low-resolution image generation module 320; wherein,

[0094] The information extraction module 310 is used to obtain the original image and extract the high-frequency information and the low-frequency information in the original image;

[0095] The low-resolution image generation module 320 is used to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.

[0096] The technical solution of the embodiment of the present disclosure,

[0097] Based on the above embodiments, the low-resolution image generation module 320 includes:

[0098] The information extraction submodule is used to process the high-frequency information based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information.

[0099] On the basis of the above embodiments, the information extraction model is an attention reversible transformation network model;

[0100] Correspondingly, the information extraction submodule includes:

[0101] An information extraction unit is used to input the high-frequency information and the low-frequency information into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, wherein the low-frequency information is an auxiliary condition for extracting content-related information from the high-frequency information.

[0102] Based on the above embodiments, the low-resolution image generation module 320 includes:

[0103] The data fusion submodule is used to perform data fusion on the content-related information and the low-frequency information in the channel dimension to obtain a low-resolution image corresponding to the original image.

[0104] Based on the above embodiments, the data fusion submodule includes:

[0105] The data fusion unit is used to determine each first channel data in the low-frequency information and at least one second channel data in the content-related information corresponding to the first channel data; and to fuse the first channel data and the second channel data having a corresponding relationship.

[0106] Based on the above embodiments, the information extraction module 310 includes:

[0107] A first information extraction unit is configured to downsample the original image to obtain low-frequency information; determine initial high-frequency information based on the original image and a high-resolution low-frequency image obtained by upsampling the low-frequency information, and rearrange spatial pixels in the initial high-frequency information to a channel dimension to obtain high-frequency information matching the low-frequency information;

[0108] or,

[0109] The second information extraction unit is used to perform filtering transformation on the original image to obtain high-frequency information and low-frequency information in the original image.

[0110] Figure 6 FIG. 1 is a schematic diagram of the structure of another image processing device provided by an embodiment of the present disclosure. Figure 6As shown, the device includes: an information extraction module 410 and an original image generation module 420; wherein,

[0111] An information extraction module 410 is used to obtain a low-resolution image and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0112] The original image generation module 420 is used to determine the high-frequency information based on the content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

[0113] Based on the above embodiments, the information extraction module 410 includes:

[0114] A first channel data determination submodule, used to determine the correspondence between each data channel in the low-frequency information and each data channel in the low-resolution image, and determine each first channel data of the low-frequency information based on each channel data in the low-resolution image;

[0115] The second channel data determination submodule is used to determine the correspondence between each data channel in the content-related information and each data channel in the low-resolution image, and determine each second channel data of the content-related information based on each channel data in the low-resolution image.

[0116] Based on the above embodiments, the first channel data determination submodule includes:

[0117] A first channel data determining unit, configured to copy the data of each channel in the low-resolution image as first channel data of a corresponding data channel in the low-frequency information;

[0118] And, the second channel data determination submodule includes:

[0119] The second channel data determining unit is used to copy the data of each channel in the low-resolution image as the second channel data of the corresponding data channel in the content-related information.

[0120] Based on the above embodiments, the device further includes:

[0121] An information acquisition module, configured to resample information in a preset data distribution corresponding to content-irrelevant information to obtain content-irrelevant information before determining the high-frequency information based on the content-related information;

[0122] Accordingly, the information extraction module 410 includes:

[0123] The high-frequency information determination submodule is used to determine the high-frequency information based on the content-related information and the resampled content-irrelevant information.

[0124] Based on the above embodiments, the high-frequency information determination submodule includes:

[0125] The high-frequency information determination unit is used to reversely input the content-related information and the content-irrelevant information into the attention reversible transformation network model to obtain the high-frequency information output by the attention reversible transformation network model.

[0126] Based on the above embodiments, the original image generation module 420 includes:

[0127] a first original image generating unit, configured to upsample the low-frequency information to obtain a high-resolution low-frequency image; perform spatial inverse rearrangement on the channel data of the high-frequency information to obtain a high-resolution high-frequency image, and obtain an original image based on the high-resolution low-frequency image and the high-resolution high-frequency image;

[0128] or,

[0129] The second original image generating unit is used to perform Haar inverse transform on the high-frequency information and the low-frequency information to obtain an original image.

[0130] The device provided by the embodiments of the present disclosure can execute the method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0132] Reference below Figure 7 , which shows an electronic device (eg, Figure 7 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0133] like Figure 7As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0134] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0135] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0136] The electronic device provided in the embodiment of the present disclosure and the holographic projection model method provided in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0137] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method provided by the above embodiments is implemented.

[0138] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0139] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0141] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0142] Acquire an original image, and extract high-frequency information and low-frequency information from the original image;

[0143] Content-related information is extracted from the high-frequency information, and the content-related information is written into the low-frequency information to obtain a low-resolution image corresponding to the original image.

[0144] or,

[0145] Acquire a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0146] The high-frequency information is determined based on the content-related information, and an original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.

[0147] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0148] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0149] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit / module does not, in some cases, constitute a limitation on the unit itself.

[0150] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0151] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0152] According to one or more embodiments of the present disclosure, [Example 1] provides an image processing method, the method comprising:

[0153] Acquire an original image, and extract high-frequency information and low-frequency information from the original image;

[0154] Content-related information is extracted from the high-frequency information, and the content-related information is written into the low-frequency information to obtain a low-resolution image corresponding to the original image.

[0155] According to one or more embodiments of the present disclosure, [Example 2] provides an image processing method, further comprising:

[0156] The extracting content-related information from the high-frequency information includes:

[0157] The high-frequency information is processed based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information.

[0158] According to one or more embodiments of the present disclosure, [Example 3] provides an image processing method, further comprising:

[0159] The information extraction model is an attention reversible transformation network model;

[0160] The processing of the high-frequency information based on the pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information includes:

[0161] The high-frequency information and the low-frequency information are input into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, wherein the low-frequency information is an auxiliary condition for extracting content-related information from the high-frequency information.

[0162] According to one or more embodiments of the present disclosure, [Example 4] provides an image processing method, further comprising:

[0163] Writing the content-related information into the low-frequency information to obtain a low-resolution image corresponding to the original image includes:

[0164] The content-related information and the low-frequency information are data-fused in the channel dimension to obtain a low-resolution image corresponding to the original image.

[0165] According to one or more embodiments of the present disclosure, [Example 5] provides an image processing method, further comprising:

[0166] The fusing the content-related information and the low-frequency information in a channel dimension includes:

[0167] Determine at least one second channel data in the content-related information corresponding to each first channel data in the low-frequency information; and perform data fusion on the first channel data and the second channel data having a corresponding relationship.

[0168] According to one or more embodiments of the present disclosure, [Example 6] provides an image processing method, further comprising:

[0169] The extracting high-frequency information and low-frequency information from the original image includes:

[0170] Downsampling the original image to obtain low-frequency information; determining initial high-frequency information based on the original image and a high-resolution low-frequency image obtained by upsampling the low-frequency information, and rearranging spatial pixels in the initial high-frequency information to a channel dimension to obtain high-frequency information matching the low-frequency information;

[0171] or,

[0172] The original image is subjected to filtering transformation to obtain high-frequency information and low-frequency information in the original image.

[0173] According to one or more embodiments of the present disclosure, [Example 7] provides an image processing method, further comprising:

[0174] Acquire a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0175] The high-frequency information is determined based on the content-related information, and an original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.

[0176] According to one or more embodiments of the present disclosure, [Example 8] provides an image processing method, further comprising:

[0177] The extracting content-related information from the low-frequency information and the high-frequency information based on the low-resolution image includes:

[0178] Determine a correspondence between each data channel in the low-frequency information and each data channel in the low-resolution image, and determine each first channel data of the low-frequency information based on each channel data in the low-resolution image;

[0179] The corresponding relationship between each data channel in the content-related information and each data channel in the low-resolution image is determined, and each second channel data of the content-related information is determined based on each channel data in the low-resolution image.

[0180] According to one or more embodiments of the present disclosure, [Example 9] provides an image processing method, including:

[0181] The determining of each first channel data of the low-frequency information based on each channel data in the low-resolution image comprises:

[0182] Copying the data of each channel in the low-resolution image as the first channel data of the corresponding data channel in the low-frequency information;

[0183] And, determining each second channel data of the content-related information based on each channel data in the low-resolution image, comprising:

[0184] The data of each channel in the low-resolution image is copied as the second channel data of the corresponding data channel in the content-related information.

[0185] According to one or more embodiments of the present disclosure, [Example 10] provides an image processing method, further comprising:

[0186] Before determining the high-frequency information based on the content-related information, the method further includes:

[0187] Resampling the information in a preset data distribution corresponding to the content-irrelevant information to obtain the content-irrelevant information;

[0188] Accordingly, determining the high-frequency information based on the content-related information includes:

[0189] The high frequency information is determined based on the content-related information and the resampled content-irrelevant information.

[0190] According to one or more embodiments of the present disclosure, [Example 11] provides an image processing method, further comprising:

[0191] The determining the high-frequency information based on the content-related information and the resampled content-irrelevant information includes:

[0192] The content-related information and the content-irrelevant information are reversely input into the attention reversible transformation network model to obtain high-frequency information output by the attention reversible transformation network model.

[0193] According to one or more embodiments of the present disclosure, [Example 12] provides an image processing method, further comprising:

[0194] The obtaining of an original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information includes:

[0195] Upsampling the low-frequency information to obtain a high-resolution low-frequency image; spatially inversely rearranging the channel data of the high-frequency information to obtain a high-resolution high-frequency image, and obtaining an original image based on the high-resolution low-frequency image and the high-resolution high-frequency image;

[0196] or,

[0197] Performing Haar inverse transform on the high-frequency information and the low-frequency information to obtain an original image.

[0198] According to one or more embodiments of the present disclosure, [Example 13] provides an image processing device, the device comprising:

[0199] An information extraction module, used to obtain an original image and extract high-frequency information and low-frequency information from the original image;

[0200] The low-resolution image generation module is used to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image.

[0201] According to one or more embodiments of the present disclosure, [Example 14] provides an image processing device, the device comprising:

[0202] An information extraction module, used to obtain a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image;

[0203] The original image generation module is used to determine the high-frequency information based on the content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

[0204] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0205] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0206] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: Acquire an original image, and extract high-frequency information and low-frequency information from the original image; Extracting content-related information from the high-frequency information, and writing the content-related information into the low-frequency information to obtain a low-resolution image corresponding to the original image; Wherein, the low-resolution image is obtained by fusing the content-related information and the low-frequency information in a channel dimension; The first channel data in the low-frequency information and the second channel data in the content-related information have a corresponding relationship, and the first channel data and the second channel data having the corresponding relationship are data fused.

2. The method according to claim 1, characterized in that The extracting content-related information from the high-frequency information includes: The high-frequency information is processed based on a pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information.

3. The method according to claim 2, characterized in that The information extraction model is an attention reversible transformation network model; The processing of the high-frequency information based on the pre-trained information extraction model to obtain content-related information and content-irrelevant information in the high-frequency information includes: The high-frequency information and the low-frequency information are input into the attention reversible transformation network model to obtain content-related information and content-irrelevant information in the high-frequency information, wherein the low-frequency information is an auxiliary condition for extracting content-related information from the high-frequency information.

4. The method according to claim 1, characterized in that: The extracting high-frequency information and low-frequency information from the original image includes: Downsampling the original image to obtain low-frequency information; determining initial high-frequency information based on the original image and a high-resolution low-frequency image obtained by upsampling the low-frequency information, and rearranging spatial pixels in the initial high-frequency information to a channel dimension to obtain high-frequency information matching the low-frequency information; or, The original image is subjected to filtering transformation to obtain high-frequency information and low-frequency information in the original image.

5. An image processing method, characterized in that: include: Acquire a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image; The low-resolution image is obtained by fusing the content-related information and the low-frequency information in a channel dimension; The first channel data in the low-frequency information and the second channel data in the content-related information have a corresponding relationship, and the low-frequency information and the content-related information are extracted from the low-resolution image based on the corresponding relationship; The high-frequency information is determined based on the content-related information, and an original image corresponding to the low-resolution image is obtained based on the fusion of the high-frequency information and the low-frequency information.

6. The method according to claim 5, characterized in that The extracting content-related information from the low-frequency information and the high-frequency information based on the low-resolution image includes: Determine a correspondence between each data channel in the low-frequency information and each data channel in the low-resolution image, and determine each first channel data of the low-frequency information based on each channel data in the low-resolution image; The corresponding relationship between each data channel in the content-related information and each data channel in the low-resolution image is determined, and each second channel data of the content-related information is determined based on each channel data in the low-resolution image.

7. The method according to claim 6, characterized in that The determining each first channel data of the low-frequency information based on each channel data in the low-resolution image includes: Copying the data of each channel in the low-resolution image as the first channel data of the corresponding data channel in the low-frequency information; And, determining each second channel data of the content-related information based on each channel data in the low-resolution image, comprising: The data of each channel in the low-resolution image is copied as the second channel data of the corresponding data channel in the content-related information.

8. The method according to claim 5, characterized in that Before determining the high-frequency information based on the content-related information, the method further includes: Resampling the information in a preset data distribution corresponding to the content-irrelevant information to obtain the content-irrelevant information; Accordingly, determining the high-frequency information based on the content-related information includes: The high frequency information is determined based on the content-related information and the resampled content-irrelevant information.

9. The method according to claim 8, characterized in that The determining the high-frequency information based on the content-related information and the resampled content-irrelevant information includes: The content-related information and the content-irrelevant information are reversely input into the attention reversible transformation network model to obtain high-frequency information output by the attention reversible transformation network model.

10. The method according to claim 5, characterized in that The obtaining of an original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information includes: Upsampling the low-frequency information to obtain a high-resolution low-frequency image; spatially inversely rearranging the channel data of the high-frequency information to obtain a high-resolution high-frequency image, and obtaining an original image based on the high-resolution low-frequency image and the high-resolution high-frequency image; or, Performing Haar inverse transform on the high-frequency information and the low-frequency information to obtain an original image.

11. An image processing device, characterized in that: include: An information extraction module, used to obtain an original image and extract high-frequency information and low-frequency information from the original image; A low-resolution image generation module, used to extract content-related information from the high-frequency information, write the content-related information into the low-frequency information, and obtain a low-resolution image corresponding to the original image; Wherein, the low-resolution image is obtained by fusing the content-related information and the low-frequency information in a channel dimension; The first channel data in the low-frequency information and the second channel data in the content-related information have a corresponding relationship, and the first channel data and the second channel data having the corresponding relationship are data fused.

12. An image processing device, characterized in that: include: An information extraction module, used to obtain a low-resolution image, and extract content-related information from low-frequency information and high-frequency information based on the low-resolution image; The low-resolution image is obtained by fusing the content-related information and the low-frequency information in a channel dimension; The first channel data in the low-frequency information and the second channel data in the content-related information have a corresponding relationship, and the low-frequency information and the content-related information are extracted from the low-resolution image based on the corresponding relationship; The original image generation module is used to determine the high-frequency information based on the content-related information, and obtain the original image corresponding to the low-resolution image based on the fusion of the high-frequency information and the low-frequency information.

13. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1 to 10.

14. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the image processing method according to any one of claims 1 to 10 when executed by a computer processor.

Images