Adaptive image preprocessing method and device based on intra block copy and medium

Abstract

The application discloses an adaptive image preprocessing method and device based on intra block copy and a medium. The application obtains a reconstructed image of a coded region, inputs the reconstructed image into a classifier for judgment processing, and obtains a judgment processing result. When the judgment processing result indicates that preprocessing is needed, the reconstructed image is subjected to image enhancement processing to obtain an enhanced image, which is beneficial to improving compression artifacts of the reconstructed image. The application obtains a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, determines a final image according to the first real probability and the second real probability to replace the reconstructed image, or obtains a first pixel mean value of the reconstructed image and a second pixel mean value of the enhanced image, and determines a final image according to the first pixel mean value and the second pixel mean value to replace the reconstructed image. The final image with better image quality can be used as a reference block, which can effectively improve prediction accuracy and coding efficiency, and can be widely applied to the field of video coding.

Description

Technical Field

[0001] This invention relates to the field of video coding, and in particular to an adaptive image preprocessing method, apparatus, and medium based on intra-frame block copying. Background Technology

[0002] In recent years, with the continuous development of computer technology, screen content video has received increasing attention in the field of video coding. Intra-Block Copying (IBC) is the best-performing and most efficient tool in screen content coding. IBC is used for intra-frame coding and belongs to the intra-frame prediction stage. When encoding the current block, IBC searches for a reference region. If a block with the same (or similar) pixel values ​​as the current coded block exists in the reference region, IBC is used to predict the current block. Therefore, the image quality of the reference block directly determines the quality of the current coded block. Traditional IBC methods reconstruct images without any filtering, resulting in severe block artifacts, ringing artifacts, and other compression artifacts. This significantly affects the quality of the reference region image blocks, leading to inaccurate matching between the reference block and the current coded block during the search for matching blocks. This increases the amount of data transmitted for block vector compensation, affecting the effectiveness of the IBC method. Furthermore, it may result in the IBC tool failing to match, forcing the use of traditional intra-frame prediction tools with very low coding efficiency, thus greatly increasing the required bitrate. Therefore, a solution is needed. Summary of the Invention

[0003] In view of this, in order to solve at least one of the above-mentioned technical problems, the object of the present invention is to provide an adaptive image preprocessing method, apparatus and medium based on intra-frame block copying to improve coding efficiency.

[0004] The technical solution adopted in the embodiments of the present invention is as follows:

[0005] Adaptive image preprocessing methods based on intra-frame block copying include:

[0006] Obtain the reconstructed image of the encoded region;

[0007] The reconstructed image is input into a classifier for judgment processing to obtain the judgment processing result;

[0008] When the judgment and processing result indicates that preprocessing is required, image enhancement processing is performed on the reconstructed image to obtain an enhanced image;

[0009] acquire a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, and determine a final image according to the first real probability and the second real probability to replace the reconstructed image, or acquire a first pixel mean value of the reconstructed image and a second pixel mean value of the enhanced image, and determine a final image according to the first pixel mean value and the second pixel mean value to replace the reconstructed image.

[0010] Further, the inputting the reconstructed image into the classifier for judgment processing to obtain a judgment processing result comprises:

[0011] inputting the reconstructed image into the classifier to segment the reconstructed image to obtain a plurality of reconstructed sub-blocks, calculating a pixel value variance of each reconstructed sub-block, and determining a number of reconstructed sub-blocks with a pixel value variance of 0, and obtaining a judgment processing result indicating that preprocessing is needed when the number is less than a preset number threshold, otherwise obtaining a judgment processing result indicating that preprocessing is not needed;

[0012] Or,

[0013] inputting the reconstructed image into the classifier to identify whether the reconstructed image contains a region to be processed, and obtaining a judgment processing result indicating that preprocessing is needed if the reconstructed image contains the region to be processed, otherwise obtaining a judgment processing result indicating that preprocessing is not needed.

[0014] Further, the image enhancement processing on the reconstructed image to obtain an enhanced image comprises:

[0015] performing first convolution on the reconstructed image and downsampling processing on a first convolution result;

[0016] performing processing on the downsampling processing result through a residual network and upsampling processing;

[0017] performing second convolution on the upsampling processing result to obtain an enhanced image.

[0018] Further, the acquiring the first real probability corresponding to the reconstructed image and the second real probability of the enhanced image comprises:

[0019] inputting the reconstructed image into a generative adversarial network model to obtain a first real probability;

[0020] inputting the enhanced image into the generative adversarial network model to obtain a second real probability;

[0021] The first real probability represents the closeness of the reconstructed image to an original image, and the second real probability represents the closeness of the enhanced image to the original image, and the original image is the reconstructed image before encoding processing.

[0022] Further, the determining the final image according to the first real probability and the second real probability comprises:

[0023] When the second real probability is greater than the first real probability, the enhanced image is determined as the final image.

[0024] Or,

[0025] When the second real probability is less than the first real probability, the reconstructed image is determined as the final image.

[0026] Further, the determining the final image according to the first pixel mean and the second pixel mean comprises:

[0027] Calculating a ratio of the second pixel mean and the first pixel mean.

[0028] When the ratio is in a preset ratio range, the enhanced image is determined as the final image, otherwise, the reconstructed image is determined as the final image.

[0029] Further, the method further comprises:

[0030] When the judgment processing result represents that the preprocessing is not needed, the reconstructed image is taken as the final image.

[0031] The embodiment of the application further provides an adaptive image preprocessing device based on intra block copy, comprising:

[0032] An acquisition module is configured to acquire a reconstructed image of a coded region.

[0033] A judgment module is configured to input the reconstructed image into a classifier to perform judgment processing and obtain a judgment processing result.

[0034] An enhancement module is configured to perform image enhancement processing on the reconstructed image to obtain an enhanced image when the judgment processing result represents that the preprocessing is needed.

[0035] A determination module is configured to acquire a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, and determine a final image according to the first real probability and the second real probability to replace the reconstructed image, or acquire a first pixel mean of the reconstructed image and a second pixel mean of the enhanced image, and determine a final image according to the first pixel mean and the second pixel mean to replace the reconstructed image.

[0036] The embodiment of the present application also provides an adaptive image preprocessing device based on intra block copy, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize the method.

[0037] The embodiment of the present application also provides a computer readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to realize the method.

[0038] The present application has the beneficial effects that: by obtaining a reconstructed image of a coded region, inputting the reconstructed image into a classifier for judgment processing to obtain a judgment processing result, when the judgment processing result represents that preprocessing is needed, performing image enhancement processing on the reconstructed image to obtain an enhanced image, which is conducive to improving the compression artifacts of the reconstructed image and repairing the distorted texture and details; obtaining a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, and determining a final image according to the first real probability and the second real probability to replace the reconstructed image, or obtaining a first pixel mean value of the reconstructed image and a second pixel mean value of the enhanced image, and determining a final image according to the first pixel mean value and the second pixel mean value to replace the reconstructed image, which is conducive to making the final image with better image quality as a reference block, effectively improving the prediction accuracy and coding efficiency, and improving the problem of inaccurate intra block copy matching caused by distorted reference blocks in the traditional intra block copy method. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 It is a step flowchart of the adaptive image preprocessing method based on intra block copy of the present application.

[0040] Figure 2 It is a video encoding flowchart of the embodiment of the present application.

[0041] Figure 3 It is a schematic diagram of step S210 of the embodiment of the present application.

[0042] Figure 4 It is a schematic diagram of the generator of the embodiment of the present application.

[0043] Figure 5 It is a training schematic diagram of the adversarial network of the embodiment of the present application. DETAILED DESCRIPTION

[0044] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of the present application.

[0045] The terms "first", "second", "third", and "fourth" and the like in the description and in the claims of the present application and the drawings are used to distinguish different objects, rather than to describe a particular order. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0046] Reference herein to "embodiments" means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor does it necessarily refer to a separate or alternative embodiment. It is explicitly and implicitly understood by those skilled in the art that embodiments described herein can be combined with other embodiments.

[0047] As shown in Figure 1 The embodiment of the present application provides an adaptive image preprocessing method based on intra block copy, comprising steps S100-S400:

[0048] S100, a reconstructed image of a coded region is obtained.

[0049] Optionally, when a current block is coded, a coded region (reference region) is searched, if there is a reference block with the same (or similar) pixel value as the current coded block in the reference region, the intra block copy method is used, the reference block is used as the prediction of the current block, and a block vector (BV) pointed to the reference block by the current block is calculated for subsequent processing, and the reference block is the reconstructed image (or the reconstructed image is part of the reference block). It should be noted that the reference block (reconstructed image) is obtained by coding the original image, that is, the original image is the reconstructed image before coding. Optionally, as shown in Figure 2As shown, the encoding process can include steps such as encoding tree unit, transformation, quantization, inverse quantization and inverse transformation, and the adaptive image preprocessing method based on intra block copy of the embodiment of the application is used to add the step of preprocessing to the reference block (reconstructed image) that can be used before the intra block copy is the core of the intra prediction link to obtain the final image to enhance the image quality, and the final image obtained after preprocessing is used as the reference of the current encoding block, which can effectively improve the prediction accuracy and encoding efficiency, improve the problem of inaccurate intra block copy matching caused by reference block distortion, the residual value is smaller, the code rate (BD-rate) of video encoding is reduced, and the peak signal-to-noise ratio (PSNR) of the overall image is improved.

[0050] S200, input the reconstructed image into the classifier for judgment processing to obtain a judgment processing result.

[0051] In the embodiment of the application, the subsequent preprocessing is performed on the reconstructed image with more text and complex texture, and the preprocessing is skipped for the region with pure color background or simple texture, unnecessary additional processing of the background region is avoided, the time consumed by the processing link can be effectively reduced, and thus the processing efficiency is improved. Optionally, the classifier includes but is not limited to a neural network binary classifier or a module unit composed of a classification algorithm.

[0052] Optionally, the step S200 includes step S210 or S220:

[0053] S210, input the reconstructed image into the classifier to segment the reconstructed image to obtain a plurality of reconstructed sub-blocks, calculate the pixel value variance of each reconstructed sub-block, and determine the number of reconstructed sub-blocks with pixel value variance of 0, when the number is less than a preset number threshold, obtain a judgment processing result indicating that preprocessing is needed, otherwise obtain a judgment processing result indicating that preprocessing is not needed.

[0054] As shown in the figure, Figure 3 Specifically, the reconstructed image is input into the classifier to segment the reconstructed image, and the reconstructed image is input into the classifier to segment the reconstructed image, wherein the number of segmentation is determined according to the reconstructed image and the actual situation, for example, assuming that the reconstructed image is a 64x64 image block, which is divided into four 32x32 reconstructed sub-blocks (i.e. Figure 3The pixel value variance of each reconstructed sub-block is calculated (only the luminance is calculated), and the number of reconstructed sub-blocks with a pixel value variance of 0 is determined. In an embodiment of the present application, the preset number threshold is 2, and in other embodiments, the number threshold can be other numbers. When the number is less than 2, that is, the number of reconstructed sub-blocks with a pixel value variance of 0 is less than 2, it is determined that the reconstructed image is a texture complex region, and a judgment result indicating that preprocessing is needed is obtained. When the number is greater than or equal to 2, that is, the number of reconstructed sub-blocks with a pixel value variance of 0 is greater than or equal to 2, it is determined that the reconstructed image is a texture simple region, and a judgment result indicating that preprocessing is not needed is obtained. In this case, the reconstructed image is taken as the final image, and no subsequent preprocessing step is needed.

[0055] In S220, the reconstructed image is input into the classifier to identify whether the reconstructed image contains a region to be processed. If the region to be processed is contained, a judgment result indicating that preprocessing is needed is obtained. Otherwise, a judgment result indicating that preprocessing is not needed is obtained.

[0056] Alternatively, a large number of images containing texture simple regions and texture complex regions can be collected in advance, and then the images are used to train a neural network binary classifier as the classifier by using a method of adding labels artificially. The trained neural network binary classifier is used to determine whether the reconstructed image needs preprocessing. Specifically, when the reconstructed image is input into the classifier, the neural network binary classifier identifies whether the reconstructed image contains a region to be processed, such as a texture complex region. If the region to be processed is contained, a judgment result indicating that preprocessing is needed is obtained. Otherwise, a judgment result indicating that preprocessing is not needed is obtained, and no subsequent preprocessing step is needed.

[0057] In S300, when the judgment result indicates that preprocessing is needed, image enhancement processing is performed on the reconstructed image to obtain an enhanced image.

[0058] Alternatively, the image enhancement processing of the reconstructed image is performed by a generator in an embodiment of the present application. The reconstructed image is input into the generator to output an enhanced image of the same size. The image enhancement processing has the following effects: 1) improving compression artifacts of the reconstructed image, repairing distorted textures and details, and effectively removing compression artifacts such as ringing effect and block effect caused by encoding; 2) relieving image distortion, repairing damaged texture information of the image, and improving the peak signal-to-noise ratio (PSNR) of the video; and 3) being conducive to making the final image closer to the original image, thus making the search and prediction process of intra block copy more accurate, thereby increasing the selection rate of the IBC mode, reducing the residual value transmitted in the IBC prediction compensation process, and thus reducing the bit number consumed by the original video for encoding, achieving the effect of reducing the video bit rate (BD-rate) and improving the coding efficiency of screen content coding.

[0059] Optionally, the generator includes, but is not limited to, a generator model based on a generative adversarial network (GAN), a convolutional neural network (CNN) model, a filtering interpolation algorithm, etc. As shown in Figure 4 In an embodiment of the present application, the neural network model of the generative adversarial network includes a first convolutional layer, a residual network (including 9 residual connection blocks (RBs), each of which includes the processing of two convolutional layers and the activation function processing between the two convolutional layers), and a second convolutional layer. Specifically, the first convolutional layer performs first convolution on the reconstructed image and downsamples the first convolution result. The downsampled result is processed by the residual network and upsampled. The upsampled result is second-convoluted to obtain the enhanced image.

[0060] S400, a first real probability corresponding to the reconstructed image is obtained, a second real probability of the enhanced image is obtained, and a final image is determined according to the first real probability and the second real probability to replace the reconstructed image, or a first pixel mean of the reconstructed image and a second pixel mean of the enhanced image are obtained, and a final image is determined according to the first pixel mean and the second pixel mean to replace the reconstructed image.

[0061] It should be noted that the process of determining the final image in step S400 is performed by the discriminator, which includes but is not limited to a convolutional neural network (CNN) model, an image quality evaluation algorithm, a discriminative network, and a discriminative algorithm. In an embodiment of the present application, the adversarial network model and the discriminative algorithm using the pixel mean are taken as examples.

[0062] Optionally, obtaining the first real probability corresponding to the reconstructed image and obtaining the second real probability of the enhanced image in step S400 include step S410:

[0063] S410, inputting the reconstructed image into the adversarial network model to obtain the first real probability, and inputting the enhanced image into the adversarial network model to obtain the second real probability.

[0064] It should be noted that the first real probability represents the closeness of the reconstructed image to the original image, and the second real probability represents the closeness of the enhanced image to the original image. The greater the first real probability, the higher the image quality of the reconstructed image, and the greater the second real probability, the higher the image quality of the enhanced image.

[0065] Optionally, determining the final image according to the first real probability and the second real probability in step S400 includes step S420:

[0066] When the second real probability is greater than the first real probability, the enhanced image is determined as the final image.

[0067] Or,

[0068] When the second real probability is less than the first real probability, the reconstructed image is determined as the final image.

[0069] In the embodiment of the present application, when the second real probability is greater than the first real probability, it is considered that the image quality of the enhanced image is higher than that of the reconstructed image, and the enhanced image is closer to the original image. At this time, the enhanced image is determined as the final image to replace the reconstructed image as the new reference block for the current coding block in the intra block copy process, so as to improve the prediction accuracy and coding efficiency. In addition, when the second real probability is less than the first real probability, it is considered that the image quality of the reconstructed image is higher than that of the enhanced image, and the reconstructed image is closer to the original image. At this time, the reconstructed image is determined as the final image to replace the reconstructed image, that is, the original reconstructed image is used as the reference for the current coding block in the intra block copy process, so that the image with higher image quality is used as the reference for the current coding block, which is beneficial to improve the prediction accuracy and coding efficiency.

[0070] Optionally, the step S400 of determining the final image according to the first pixel mean value and the second pixel mean value comprises steps S430-S440:

[0071] S430, calculating the ratio of the second pixel mean value to the first pixel mean value.

[0072] S440, when the ratio is in a preset ratio range, the enhanced image is determined as the final image, otherwise the reconstructed image is determined as the final image.

[0073] Optionally, the preset ratio range includes but is not limited to 0.85-1.15. When the ratio of the second pixel mean value to the first pixel mean value is in the range of 0.85-1.15, it is considered that the image quality of the enhanced image is higher than that of the reconstructed image. At this time, the enhanced image is determined as the final image to replace the reconstructed image as the new reference block for the current coding block in the intra block copy process. When the ratio of the second pixel mean value to the first pixel mean value is not in the range of 0.85-1.15, it is considered that the image quality of the reconstructed image is higher than that of the enhanced image. At this time, the reconstructed image is determined as the final image to replace the reconstructed image, that is, the original reconstructed image is used as the reference for the current coding block in the intra block copy process, so that the image with higher image quality is used as the reference for the current coding block.

[0074] It should be noted that using the image with higher image quality as the final image is beneficial to improve the stability of the preprocessing and the robustness of the entire model, and prevent negative optimization.

[0075] Optionally, after the final image is determined, the preprocessing process is continued to end, and the final image is output for intra block prediction reference and compensation. The regular encoding process is continued until the coding unit (CU) of the next block is reconstructed, a new current block is determined, and the step S100 is returned.

[0076] AsFigure 5 As shown, the above-mentioned adversarial network model can be obtained by the following steps:

[0077] S101: The encoding processing outputs a training reconstructed image without filtering processing, and a filtering module is closed to output a training original image (a real image) actually in an intra prediction link;

[0078] S102: The image in the above step is respectively cut into 64*64 small blocks, and YUV format is converted into RGB format picture to adapt to a neural network, and the picture is input into the network training as a training set;

[0079] S103: Training and parameter adjustment of a generative adversarial network (GAN) model are generated, the adversarial network model includes a first generator and a first discriminator which are against each other, the first generator can only see the training reconstructed image in the training data set, and generates an enhanced image based on the training reconstructed image, and the first discriminator can see the image generated by the generator and the training original image, but the first discriminator does not know which one is generated by the generator, the first discriminator gives a probability that the two images are real images respectively through identification, and the probability value is fed back to the generator to stimulate the first generator to generate an image with higher quality. When the adversarial training converges, the GAN network is in a Nash equilibrium state, that is, the image generated by the first generator is sufficient to confuse the real and the false (the quality of the corresponding result is an enhanced image (denoted as a training enhanced image) close to the training original image), and the first discriminator outputs a result close to 0.5 because it cannot judge which image is real and which image is generated, and an ideal generator is obtained; the training strategy is adjusted to make the training tilt to the discriminator (usually, the learning rate of the discriminator is adjusted), and a converged first discriminator is obtained, the first discriminator does not correspond to the first generator in the first training, and the performance of the first discriminator is stronger than that of the first discriminator in the first training, and the first discriminator can better give the probability that the image is a real image (the final performance is that the higher the image quality is, the closer the output real probability value is to 1. Finally, a first generator and a first discriminator with sufficient superior performance are obtained, and the training is completed.

[0080] The embodiment of the application further provides an adaptive image preprocessing device based on intra block copy, comprising:

[0081] The acquisition module is configured to acquire a reconstructed image of a coded region;

[0082] The judgment module is configured to input the reconstructed image into a classifier for judgment processing to obtain a judgment processing result;

[0083] The enhancement module is configured to, when the judgment processing result represents that preprocessing is needed, perform image enhancement processing on the reconstructed image to obtain an enhanced image;

[0084] The determining module is used for obtaining the first real probability corresponding to the reconstructed image and the second real probability of the enhanced image, determining the final image to replace the reconstructed image according to the first real probability and the second real probability, or obtaining the first pixel mean value of the reconstructed image and the second pixel mean value of the enhanced image, and determining the final image to replace the reconstructed image according to the first pixel mean value and the second pixel mean value.

[0085] The contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically realize the functions same as those of the method embodiments, and achieve the beneficial effects same as those of the method embodiments.

[0086] The embodiment of the application further provides another adaptive image preprocessing device based on intra block copy, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to realize the adaptive image preprocessing method based on intra block copy of the foregoing embodiment.

[0087] The contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically realize the functions same as those of the method embodiments, and achieve the beneficial effects same as those of the method embodiments.

[0088] The embodiment of the application further provides a computer readable storage medium, the storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to realize the adaptive image preprocessing method based on intra block copy of the foregoing embodiment.

[0089] The embodiment of the application further provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer equipment reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer equipment executes the adaptive image preprocessing method based on intra block copy of the foregoing embodiment.

[0090] The terms "first", "second", "third", "fourth" and the like in the description of this application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover the embodiments of the application whether or not the embodiments are described with the use of these terms. The use of the terms "first", "second", "third", "fourth", and the like, if any, are not intended to imply a chronological or sequential order, but are used to distinguish one element from another. Unless otherwise specified, the use of the ordinal adjectives "first", "second", "third", "fourth", and the like, if any, in the description of this application and in the claims to indicate that certain claims are intended to cover a variety of embodiments and / or aspects of the application, and do not necessarily indicate a chronological or sequential order to processes, operations, acts, events, or the like, unless explicitly directed by the passage. Moreover, the terms "comprise", "have", "contain", and "include" and their conjugates, as used herein, are used in their open-ended, non-limiting sense to encompass modifications, alterations, additions, and / or improvements that can be made to the described embodiments that are within the scope of the disclosure.

[0091] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or", used to describe the relationship between associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0092] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. The division of the units is merely logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0093] The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially or the part contributing to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions used to cause a computer device (such as a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which are all various types of media that can store programs.

[0094] The above, the above embodiments are merely used to describe the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of adaptive image pre-processing based on intra block copy, characterized in that, The method comprises the following steps: obtaining a reconstructed image of an encoded region; inputting the reconstructed image into a classifier for judgment processing to obtain a judgment processing result; when the judgment processing result indicates that preprocessing is needed, performing image enhancement processing on the reconstructed image to obtain an enhanced image; obtaining a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, and determining a final image to replace the reconstructed image according to the first real probability and the second real probability, or obtaining a first pixel mean value of the reconstructed image and a second pixel mean value of the enhanced image, and determining a final image to replace the reconstructed image according to the first pixel mean value and the second pixel mean value; the determination of the final image according to the first real probability and the second real probability comprises: when the second real probability is greater than the first real probability, the enhanced image is determined as the final image; or, when the second real probability is less than the first real probability, the reconstructed image is determined as the final image; the determination of the final image according to the first pixel mean value and the second pixel mean value comprises: calculating the ratio of the second pixel mean value to the first pixel mean value; when the ratio is within a preset ratio range, the enhanced image is determined as the final image, otherwise the reconstructed image is determined as the final image; the obtaining of the first real probability corresponding to the reconstructed image and the second real probability of the enhanced image comprises: inputting the reconstructed image into a generative adversarial network model to obtain the first real probability; inputting the enhanced image into the generative adversarial network model to obtain the second real probability; the first real probability represents the closeness of the reconstructed image to an original image, and the second real probability represents the closeness of the enhanced image to the original image, the original image being the reconstructed image before encoding processing.

2. The adaptive image pre-processing method based on intra block copy according to claim 1, characterized in that: the inputting of the reconstructed image into the classifier for judgment processing to obtain a judgment processing result comprises: inputting the reconstructed image into the classifier to segment the reconstructed image to obtain a plurality of reconstructed subblocks, calculating the pixel value variance of each reconstructed subblock, and determining the number of reconstructed subblocks with a pixel value variance of 0, when the number is less than a preset number threshold, obtaining a judgment processing result indicating that preprocessing is needed, otherwise obtaining a judgment processing result indicating that preprocessing is not needed; or, inputting the reconstructed image into the classifier to identify whether the reconstructed image contains a region to be processed, if the reconstructed image contains the region to be processed, obtaining a judgment processing result indicating that preprocessing is needed, otherwise obtaining a judgment processing result indicating that preprocessing is not needed.

3. The adaptive image pre-processing method based on intra block copy according to claim 1, characterized in that: the image enhancement processing on the reconstructed image to obtain an enhanced image comprises: performing first convolution on the reconstructed image and downsampling the first convolution result; processing the downsampling result through a residual network and upsampling the result; performing second convolution on the upsampling result to obtain the enhanced image.

4. The adaptive image pre-processing method based on intra block copy according to claim 1, characterized in that: The method further comprises: when the judgment processing result indicates that preprocessing is not needed, taking the reconstructed image as the final image.

5. An apparatus for implementing the intra block copy based adaptive image preprocessing method according to any one of claims 1-4, characterized in that, ​ An acquisition module is configured to acquire a reconstructed image of a coded region; A determination module is configured to acquire a first real probability corresponding to the reconstructed image and a second real probability of the enhanced image, and determine a final image to replace the reconstructed image according to the first real probability and the second real probability, or acquire a first pixel mean value of the reconstructed image and a second pixel mean value of the enhanced image, and determine a final image to replace the reconstructed image according to the first pixel mean value and the second pixel mean value. The determination of the final image according to the first real probability and the second real probability includes: When the second real probability is greater than the first real probability, the enhanced image is determined as the final image; or When the second real probability is less than the first real probability, the reconstructed image is determined as the final image. The determination of the final image according to the first pixel mean value and the second pixel mean value includes: A ratio of the second pixel mean value to the first pixel mean value is calculated; When the ratio is within a preset ratio range, the enhanced image is determined as the final image, otherwise the reconstructed image is determined as the final image. A processor and a memory are included, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the method in any one of claims 1-4. The storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the method in any one of claims 1-4. ​ 6. An adaptive image preprocessing apparatus based on intra-frame block copying, characterized in that, ​ 7. A computer readable storage medium characterized in that, ​

Images