An image processing method, apparatus, device and storage medium

Abstract

Embodiments of the present disclosure provide an image processing method, device and equipment and a storage medium. The method comprises: determining a target down-sampling ratio corresponding to a first image based on an original size of the first image and a target size after processing; determining a target down-sampling network model corresponding to the first image based on the target down-sampling ratio, at least one pre-trained down-sampling network model and a preset down-sampling ratio corresponding to the down-sampling network model; determining a second image satisfying a preset down-sampling condition based on the preset down-sampling ratio corresponding to the target down-sampling network model, the target down-sampling ratio and the first image; and performing down-sampling processing on the second image based on the target down-sampling network model to obtain a target image with the target size. Through the technical solution of the embodiments of the present disclosure, the quality of the down-sampled image can be improved, and the image can be down-sampled to any size, thereby improving the flexibility of image down-sampling.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more particularly to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of computer technology, image downsampling is often necessary to reduce image size, making it fit the screen size or generating thumbnails. Currently, image downsampling is typically performed using image interpolation, for example, processing pixels within each window of the original image and mapping them to individual pixels in the target image, thus achieving image downsampling. However, this downsampling method is relatively coarse and cannot effectively guarantee the quality of the downsampled image. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, device, and storage medium to improve the quality of downsampled images and to downsample images to any size, thereby increasing the flexibility of image downsampling.

[0004] In a first aspect, embodiments of this disclosure provide an image processing method, including:

[0005] Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image;

[0006] Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, the target downsampling network model corresponding to the first image is determined.

[0007] Based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image, a second image that meets the preset downsampling conditions is determined;

[0008] Based on the target downsampling network model, the second image is downsampled to obtain a target image with the target size.

[0009] Secondly, embodiments of this disclosure also provide an image processing apparatus, comprising:

[0010] The downsampling ratio determination module is used to determine the target downsampling ratio of the first image based on the original size of the first image to be processed and the target size after processing.

[0011] The network model determination module is used to determine the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model.

[0012] The second image determination module is used to determine a second image that meets the preset downsampling conditions based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image.

[0013] The downsampling processing module is used to perform downsampling processing on the second image based on the target downsampling network model to obtain a target image with the target size.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0015] One or more processors;

[0016] Storage device for storing one or more programs.

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any of the embodiments of this disclosure.

[0018] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method as described in any of the embodiments of this disclosure.

[0019] In this embodiment, a target downsampling ratio for the first image is determined based on its original size and the target size after processing. Then, based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio for each downsampling network model, a target downsampling network model for the first image is determined from the at least one downsampling network model. Finally, based on the preset downsampling ratio of the target downsampling network model, the target downsampling ratio, and the first image, a second image satisfying preset downsampling conditions is determined. This allows the target downsampling network model to be directly used to downsample the second image at the corresponding preset downsampling ratio, obtaining a target image with the target size. By using a pre-trained target downsampling network model for deep learning downsampling processing, the quality of the downsampled image can be improved. Furthermore, by determining a matching target downsampling network model and a second image satisfying the preset downsampling conditions, images can be downsampled to any size, improving the flexibility of image downsampling. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this disclosure;

[0022] Figure 2 This is a schematic flowchart of another image processing method provided in an embodiment of this disclosure;

[0023] Figure 3 This is an example of the training process of a downsampling network model involved in an embodiment of this disclosure;

[0024] Figure 4 This is a schematic flowchart of another image processing method provided in the embodiments of this disclosure;

[0025] Figure 5 This is an example of the architecture of a target downsampling network model with a preset downsampling factor of an integer multiple, as disclosed in this embodiment.

[0026] Figure 6 This is an example of the architecture of a target downsampling network model with a preset downsampling factor of fractional factor, as disclosed in this embodiment.

[0027] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0029] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0030] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0031] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0032] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0033] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0035] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0036] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the case of downsampling an image to an arbitrary specified size. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a PC, or a server.

[0037] like Figure 1 As shown, the image processing method specifically includes the following steps:

[0038] S110. Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image.

[0039] Here, the first image can refer to the original image to be processed. A single image to be processed can be used as the first image for individual downsampling, or each frame of the video to be processed can be used as the first image for video downsampling. The first image can be a 3-channel RGB (Red, Green, Blue) color image. The original size refers to the inherent size of the first image. The target size refers to the specified size to which the first image needs to be downsampled. The target size can be any size. The target size is smaller than the original size. The target downsampling factor can refer to the factor by which the first image needs to be downsampled; it can be any value greater than 1. The target downsampling factor can be an integer factor or a fractional factor (i.e., a decimal factor).

[0040] Specifically, the original size of the first image to be processed can be divided by the target size, and the result of this division is determined as the target downsampling factor for the first image. For example, if the target downsampling factor is an integer multiple of 2, it indicates that the first image needs to be downsampled by 2 times. If the target downsampling factor is a fractional multiple of 3 / 2, it indicates that the first image needs to be downsampled by 1.5 times.

[0041] S120. Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, determine the target downsampling network model corresponding to the first image.

[0042] The downsampling network model can be a deep learning network model used to perform a preset downsampling ratio on the image. The preset downsampling ratio can be a pre-defined downsampling ratio that the downsampling network model can achieve. There can be one or more downsampling network models. Each downsampling network model is pre-trained based on sample images to ensure the quality of the downsampled image. Each downsampling network model can only achieve a fixed downsampling ratio. Different downsampling network models correspond to different preset downsampling ratios to achieve different downsampling ratios. The target downsampling network model can be the downsampling network model that best matches the first image. For example, if the preset downsampling ratio of a certain downsampling network model is N, and the image size input to the downsampling network model is (W, H), then the image size output by the downsampling network model will be (W / N, H / N), or (round(W / N), round(H / N)), where round is the rounding function.

[0043] Specifically, if only one downsampling network model is pre-trained, meaning only one downsampling network model currently exists, then this downsampling network model can be directly used as the target downsampling network model for the first image. If at least two downsampling network models are pre-trained, meaning at least two downsampling network models currently exist, then the preset downsampling ratio corresponding to each downsampling network model can be compared with the target downsampling ratio. Based on the comparison results, the target downsampling network model can be determined from the at least two downsampling network models, so as to use the target downsampling network model for more accurate image downsampling, thereby ensuring the quality of the downsampled image.

[0044] For example, S120 may include: determining the ratio difference between the target downsampling ratio and the preset downsampling ratio corresponding to each downsampling network model obtained through pre-training; and determining the target downsampling network model corresponding to the first image based on the ratio difference corresponding to each downsampling network model.

[0045] Specifically, the target downsampling factor is subtracted from the preset downsampling factor corresponding to each downsampling network model to obtain the downsampling factor difference for each downsampling network model. The downsampling network model with the smallest downsampling factor difference is determined as the target downsampling network model, thus the downsampling network model closest to the target downsampling factor is used as the target downsampling network model. The smaller the downsampling factor difference between the preset downsampling factor corresponding to the target downsampling network model and the target downsampling factor, the higher the quality of the downsampled image.

[0046] S130. Based on the preset downsampling factor, the target downsampling factor, and the first image corresponding to the target downsampling network model, determine the second image that meets the preset downsampling conditions.

[0047] Here, the preset downsampling condition refers to a pre-set condition that enables downsampling processing using the target downsampling network model. The second image can be an image that can be downsampled using the target downsampling network model. For example, the preset downsampling condition could be that the ratio of the size of the second image to the target size is a preset downsampling factor corresponding to the target downsampling network model, so that an image of the target size can be obtained after downsampling using the target downsampling network model.

[0048] Specifically, the preset downsampling ratio corresponding to the target downsampling network model can be compared with the target downsampling ratio, and the comparison result can be used to determine whether the first image meets the preset downsampling conditions. If the preset downsampling conditions are met, the first image is determined to be the second image. If the preset downsampling conditions are not met, the first image is preprocessed and sampled to obtain the second image that meets the preset downsampling conditions.

[0049] For example, S130 may include: if the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio, then the first image is determined as the second image that satisfies the preset downsampling condition; if the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, then the first image is preprocessed and sampled based on the preset downsampling ratio corresponding to the target downsampling network model and the target size to determine the second image that satisfies the preset downsampling condition.

[0050] Specifically, when the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio, it indicates that the target downsampling network model can be directly used to downsample at the target downsampling ratio. In this case, the first image can be determined as the second image that meets the preset downsampling conditions. When the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, it indicates that the target downsampling network model cannot be directly used to downsample at the target downsampling ratio. In this case, it is necessary to perform preprocessing upsampling or downsampling on the first image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size to obtain the second image that meets the preset downsampling conditions.

[0051] S140. Based on the target downsampling network model, the second image is downsampled to obtain a target image with the target size.

[0052] Specifically, a second image that meets the preset downsampling conditions can be directly input into a target downsampling network model for downsampling processing. The target downsampling network model performs downsampling processing on the input second image at a corresponding preset downsampling ratio to obtain a target image with the target size and outputs it, thereby achieving downsampling of the first image to the target size. Furthermore, by using a pre-trained target downsampling network model for deep learning downsampling processing, the quality of the downsampled image is improved.

[0053] The technical solution of this disclosure determines a target downsampling ratio for the first image based on its original size and the target size after processing. Then, based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio for each downsampling network model, a target downsampling network model for the first image is determined from the at least one downsampling network model. Finally, based on the preset downsampling ratio of the target downsampling network model, the target downsampling ratio, and the first image, a second image satisfying a preset downsampling condition is determined. This allows the target downsampling network model to be directly used to downsample the second image at the corresponding preset downsampling ratio, obtaining a target image with the target size. By using a pre-trained target downsampling network model for deep learning downsampling processing, the quality of the downsampled image can be improved. Furthermore, by determining a matching target downsampling network model and a second image satisfying the preset downsampling condition, images can be downsampled to any size, improving the flexibility of image downsampling.

[0054] Based on the above technical solution, the training process of each downsampling network model may include: using the preset downsampling ratio corresponding to the downsampling network model as the upsampling ratio to upsample the sample image to obtain an upsampled image; inputting the upsampled image into the downsampling network model to be trained for downsampling processing to obtain the output image of the downsampling network model; determining the training error based on the output image and the sample image, and backpropagating the training error to the downsampling network model to be trained for network parameter adjustment, until the preset convergence condition is reached, at which point the training of the downsampling network model is considered complete.

[0055] Specifically, each downsampling network model can be trained independently to ensure that each model accurately performs downsampling at the corresponding preset downsampling rate. Sample images can be high-resolution images to improve model training performance.

[0056] For example, Figure 2 An example of a downsampling network model training process is given. Figure 2As shown, image interpolation methods, such as bicubic interpolation or Lanczos orthogonal similarity transformation, can be used. The preset downsampling factor N corresponding to the downsampling network model can be used as the upsampling factor to upsample the sample image by a factor of N, resulting in an upsampled image. This increases the resolution of the sample image by a factor of N, resulting in a high-resolution degraded upsampled image. Since upsampling in image interpolation is equivalent to a low-pass filter, the subjective visual quality of the upsampled image is lower than that of the original sample image. The upsampled image is then input into the downsampling network model to be trained for downsampling at the corresponding preset downsampling factor, obtaining the output image of the downsampling network model. Because the output image is the sample image that has undergone N-fold upsampling and then N-fold downsampling, the output image has the same size (i.e., the same image resolution) as the original sample image. Therefore, a preset loss function can be used to determine the training error between the output image and the sample image. For example, the training error could be: or ,in, To output the image, For sample images, For pixels in the image, This represents the number of pixels in the image. The training error is backpropagated to the downsampling network model to be trained for network parameter adjustment until a preset convergence condition is met, such as the number of iterations equaling a preset number, or the training error becoming stable. At this point, the training of the downsampling network model is considered complete. The downsampling network parameters are then optimized using algorithms such as stochastic gradient descent to obtain a downsampling network model with better downsampling performance.

[0057] It should be noted that during the training process of the downsampling network model, by determining the training error between the output image and the sample image, the distortion constraint of the output image of the downsampling network model can be directly applied to the output image of the downsampling network model to make the output image closer to the original sample image, thereby ensuring the quality of the output image of the downsampling network model and effectively improving the clarity of the downsampled image.

[0058] Figure 3 This is a flowchart illustrating another image processing method provided in this embodiment. Based on the above-disclosed embodiments, this embodiment optimizes the step of "preprocessing and sampling the first image based on the preset downsampling ratio and target size corresponding to the target downsampling network model to determine the second image that meets the preset downsampling conditions." Explanations of terms that are the same as or corresponding to those in the above-disclosed embodiments are not repeated here.

[0059] like Figure 3 As shown, the image processing method specifically includes the following steps:

[0060] S310. Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image.

[0061] S320. Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, determine the target downsampling network model corresponding to the first image.

[0062] S330. Check whether the preset downsampling factor corresponding to the target downsampling network model is equal to the target downsampling factor. If yes, proceed to step S340; otherwise, proceed to step S350.

[0063] S340. Determine the first image as the second image that meets the preset downsampling conditions, and execute step S370.

[0064] Specifically, when the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio, it indicates that the target downsampling network model can be directly used to downsample at the target downsampling ratio. In this case, the first image can be directly determined as the second image that meets the preset downsampling conditions.

[0065] S350. Based on the preset downsampling ratio and target size corresponding to the target downsampling network model, determine the size of the preprocessed intermediate image.

[0066] The intermediate image size can refer to the image size that meets the preset downsampling conditions.

[0067] Specifically, when the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, it indicates that the target downsampling network model cannot be used directly to downsample at the target downsampling ratio. In this case, the intermediate image size that meets the preset downsampling conditions can be determined based on the preset downsampling ratio and target size corresponding to the target downsampling network model.

[0068] For example, S350 may include: multiplying the target size by a preset downsampling factor corresponding to the target downsampling network model, and using the multiplication result as the preprocessed intermediate image size.

[0069] Specifically, if the target size after downsampling is height H t and width W t If the preset downsampling factor corresponding to the target downsampling network model is N, then the size of the preprocessed intermediate image is H. t ×N and W t ×N, or round(H) t ×N) and round(W tThe intermediate image size can be downsampled by a factor of N to obtain the target size.

[0070] S360. Perform preprocessing sampling on the first image to determine a second image with intermediate image size, and execute step S370.

[0071] Specifically, based on image interpolation methods, such as bicubic interpolation, orthogonal similarity transformation (Lanczos method), nearest neighbor interpolation, bilinear interpolation, etc., the first image can be preprocessed by upsampling or downsampling to obtain a second image with the size of the intermediate image.

[0072] It should be noted that the size of the intermediate image can be larger or smaller than the original size of the first image. When the size of the intermediate image is larger than the original size of the first image, the first image is preprocessed and upsampled using image interpolation to obtain a second image with that intermediate image size. When the size of the intermediate image is smaller than the original size of the first image, the first image is preprocessed and downsampled using image interpolation to obtain a second image with that intermediate image size.

[0073] S370. Based on the target downsampling network model, the second image is downsampled to obtain a target image with the target size.

[0074] The technical solution of this disclosure, when the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, determines the size of the preprocessed intermediate image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size, and performs preprocessing sampling on the first image to determine the second image with the intermediate image size. Thus, by combining preprocessing sampling and the target downsampling network model, the first image can be downsampled to the target size, thereby achieving downsampling of arbitrary size while ensuring the quality of the downsampled image.

[0075] Figure 4 This is a flowchart illustrating another image processing method provided in this disclosure. Based on the above-described embodiments, this disclosure provides a detailed description of the specific architecture of the target downsampling network model. Explanations of terms that are the same as or corresponding to those in the above-described embodiments are not repeated here.

[0076] like Figure 4 As shown, the image processing method specifically includes the following steps:

[0077] S410. Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image.

[0078] S420. Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, determine the target downsampling network model corresponding to the first image.

[0079] S430. Based on the preset downsampling factor, the target downsampling factor, and the first image corresponding to the target downsampling network model, determine the second image that meets the preset downsampling conditions.

[0080] S440. Input the second image into the pixel rearrangement sub-model in the target downsampling network model to perform downsampling and rearrangement of image pixels, and obtain a first feature map with the target size.

[0081] The pixel rearrangement sub-model can be a network module used to rearrange and recombine image pixels to achieve image downsampling. The network architecture of the pixel rearrangement sub-model differs depending on whether the preset downsampling rate is an integer multiple or a fractional multiple. The network architecture of the pixel rearrangement sub-model is the same for different integer multiples. The network architecture of the pixel rearrangement sub-model is also the same for different fractional multiples. The first feature map can refer to a feature image with the same size as the target image.

[0082] Specifically, the pixel rearrangement sub-model in the target downsampling network model can downsample and rearrange the pixels of the input second image, that is, downsample the second image at a preset downsampling rate to obtain a first feature map with the target size. It should be noted that by using the pixel rearrangement sub-model for image downsampling, the amount of video memory used remains unchanged, avoiding the use of additional video memory and improving image processing efficiency and stability. The width and height of the first feature map are the same as the width and height of the target image, but the number of channels in the first feature map is different from the number of channels in the target image.

[0083] For example, see Figure 5 When the preset downsampling factor corresponding to the target downsampling network model is an integer multiple, the pixel rearrangement sub-model may include: a first pixel inverse rearrangement module. The first pixel inverse rearrangement module can achieve image downsampling by increasing the number of channels and decreasing the spatial resolution. For example, the first pixel inverse rearrangement module can be a PixelUnshuffle layer.

[0084] In this module, the channel amplification factor is equal to the preset downsampling factor corresponding to the target downsampling network model. For example, the first pixel inverse rearrangement module can rearrange an image of (C, H×r1, W×r1) into (C×r1) 2 The feature maps of H and W are used to increase the number of channels and reduce the spatial resolution, but the memory usage remains unchanged. The channel amplification factor r1 is equal to the preset downsampling factor corresponding to the target downsampling network model.

[0085] Specifically, see Figure 5 When the preset downsampling factor corresponding to the target downsampling network model is an integer multiple, S440 may include: inputting the second image into the first pixel inverse rearrangement module to perform downsampling rearrangement of image pixels to obtain a first feature map with the target size.

[0086] For example, see Figure 6 When the preset downsampling factor for the target downsampling network model is a fractional downsampling factor, the pixel rearrangement sub-model can include: a second pixel inverse rearrangement module, a first convolutional module, and a pixel rearrangement module. The second pixel inverse rearrangement module can downsample the image by increasing the number of channels and decreasing the spatial resolution. The first convolutional module can perform convolution processing on the feature map output by the second pixel inverse rearrangement module, thereby improving the quality of the downsampled image. For example, the first convolutional module can include a convolutional layer with a kernel size of 3x3, a stride of 1, and padding of 1. The pixel rearrangement module can upsample the image by decreasing the number of channels and increasing the spatial resolution. For example, the pixel rearrangement module can be a pixel shuffle layer. The pixel inverse rearrangement module and the pixel rearrangement module are inverse operations of each other. The pixel rearrangement module can perform (C×r3) pixel rearrangement on ... 2 The image (C, H, W) is rearranged and converted into a (C, H×r3, W×r3) feature map, thereby reducing the number of channels and increasing the spatial resolution, but the memory usage remains unchanged.

[0087] In this module, the channel amplification factor r2 in the second pixel inversion rearrangement module and the channel reduction factor r3 in the pixel rearrangement module are determined based on the preset downsampling factor corresponding to the target downsampling network model. Specifically, when the preset downsampling factor corresponding to the target downsampling network model is a fractional factor, such as X / Y, then the numerator X can be determined as the channel amplification factor r2 in the second pixel inversion rearrangement module, and the denominator Y as the channel reduction factor r3 in the pixel rearrangement module. By combining the second pixel inversion rearrangement module and the pixel rearrangement module, fractional downsampling can be achieved.

[0088] Specifically, see Figure 6 When the preset downsampling factor corresponding to the target downsampling network model is a fractional factor, S440 may include: inputting the second image into the second pixel inverse rearrangement module to perform downsampling rearrangement of image pixels to obtain a downsampled feature map; inputting the downsampled feature map into the first convolution module to perform convolution processing to obtain a convolutional feature map; and inputting the convolutional feature map into the pixel rearrangement module to perform upsampling rearrangement of pixels to obtain an upsampled first feature map, thereby achieving fractional downsampling.

[0089] S450. Input the first feature map into the convolution processing sub-model in the target downsampling network model to perform feature convolution processing to obtain a target image with the target size.

[0090] The convolutional processing sub-model is a network module used to perform convolutional processing on the feature maps to ensure the quality of image processing. For example, the convolutional processing sub-model may include one or more convolutional layers. Utilizing multiple convolutional layers can further guarantee the quality of image processing.

[0091] Specifically, the convolutional processing sub-model in the target downsampling network model performs depth feature convolution processing on the input first feature map, thereby obtaining a higher quality target image with the target size, effectively ensuring the image quality after downsampling.

[0092] For example, see Figure 5 or Figure 6 The convolutional processing sub-model can include a second convolutional module, a third convolutional module, and a fourth convolutional module. Each of the second, third, and fourth convolutional modules can include one or more convolutional layers. The second, third, and fourth convolutional modules perform convolutional processing at different locations to further improve the convolutional processing effect.

[0093] For example, both the second and fourth convolutional modules can include a convolutional layer with a kernel size of 3x3, a stride of 1, and padding of 1. The third convolutional module can include three convolutional layers with kernel size of 3x3, a stride of 1, and padding of 1. The second and third convolutional modules also include an activation function layer, such as a LeakyReLU activation function layer with a negative axis slope of 0.2. The fourth convolutional module does not include an activation function layer to avoid limiting expressive power. The output channel of the fourth convolutional module can be 3, meaning the output target image is an RGB image with 3 channels.

[0094] For example, see Figure 5 or Figure 6 S450 may include: inputting a first feature map into a second convolutional layer for convolution processing to obtain a processed second feature map; inputting the second feature map into a third convolutional layer for convolution processing to obtain a processed third feature map; and inputting the second and third feature maps into a fourth convolutional layer for convolution processing to obtain a target image with the target size. Alternatively, the second and third feature maps may be superimposed, and the superimposed feature map may be input into the fourth convolutional layer for convolution processing to obtain the target image with the target size.

[0095] For example, see Figure 5When the preset downsampling factor corresponding to the target downsampling network model is an integer multiple of 2, the second image with size (3, 1080, 1920) is processed by the first pixel inverse rearrangement module (channel magnification factor r1=2) to obtain a first feature map with size (12, 540, 960). This first feature map is then processed by the second convolution module to obtain a second feature map with size (16, 540, 960). This second feature map is then processed by the third convolution module to obtain a third feature map with size (16, 540, 960). The second and third feature maps are then superimposed and processed by the fourth convolution module to obtain a target image with size (3, 540, 960), thus downsampling the second image with size (1080, 1920) by a factor of 2 to obtain a target image with size (540, 960).

[0096] For example, see Figure 6 When the preset downsampling factor corresponding to the target downsampling network model is a fractional downsampling factor of 1.5, the second image with size (3, 1080, 1920) is processed by the second pixel inverse rearrangement module (channel magnification factor r2=3) to obtain a feature map with size (27, 360, 640). This feature map is then processed by the first convolution module to obtain a feature map with size (36, 360, 640). This feature map is then processed by the pixel rearrangement module (channel reduction factor r3=2) to obtain a first feature map with size (9, 720, 1280). This first feature map is then processed by the second convolution module to obtain a second feature map with size (9, 720, 1280). This second feature map is then processed by the third convolution module to obtain a third feature map with size (9, 720, 1280). The second and third feature maps are superimposed and then passed through the fourth convolution module to obtain a target image with a size of (3,720,1280). This results in the second image with a size of (1080,1920) being downsampled by 1.5 times to obtain a target image with a size of (720,1280).

[0097] The technical solution of this disclosure utilizes a pixel rearrangement sub-model and a convolution processing sub-model to perform downsampling rearrangement of image pixels and convolution processing of features on the second image. This ensures that the amount of video memory occupied remains unchanged while achieving downsampling at a preset downsampling ratio, avoiding the use of additional video memory and improving image processing efficiency and stability.

[0098] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure, as shown below. Figure 7 As shown, the device specifically includes: a downsampling ratio determination module 710, a network model determination module 720, a second image determination module 730, and a downsampling processing module 740.

[0099] The downsampling ratio determination module 710 is used to determine the target downsampling ratio corresponding to the first image based on the original size and the target size after processing of the first image to be processed; the network model determination module 720 is used to determine the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio corresponding to the downsampling network model; the second image determination module 730 is used to determine the second image that meets the preset downsampling conditions based on the preset downsampling ratio corresponding to the target downsampling network model, the target downsampling ratio, and the first image; and the downsampling processing module 740 is used to perform downsampling processing on the second image based on the target downsampling network model to obtain a target image with the target size.

[0100] The technical solution provided in this disclosure determines the target downsampling ratio of the first image based on its original size and the target size after processing. Then, based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio for each downsampling network model, a target downsampling network model corresponding to the first image is determined from at least one downsampling network model. Finally, based on the preset downsampling ratio of the target downsampling network model, the target downsampling ratio, and the first image, a second image satisfying preset downsampling conditions is determined. This allows the target downsampling network model to be directly used to downsample the second image at the corresponding preset downsampling ratio, obtaining a target image with the target size. By using a pre-trained target downsampling network model for deep learning downsampling processing, the quality of the downsampled image can be improved. Furthermore, by determining a matching target downsampling network model and a second image satisfying the preset downsampling conditions, images can be downsampled to any size, improving the flexibility of image downsampling.

[0101] Based on the above technical solution, the network model determination module 720 is specifically used for:

[0102] Determine the ratio difference between the target downsampling ratio and the preset downsampling ratio corresponding to each pre-trained downsampling network model; based on the ratio difference corresponding to each downsampling network model, determine the target downsampling network model corresponding to the first image.

[0103] Based on the above technical solutions, the second image determination module 730 includes:

[0104] The first determining unit is configured to determine the first image as a second image that satisfies the preset downsampling condition if the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio.

[0105] The second determining unit is configured to preprocess and sample the first image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size if the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, thereby determining a second image that meets the preset downsampling conditions.

[0106] Based on the above technical solutions, the second determining unit includes:

[0107] The intermediate image size determination subunit is used to determine the size of the preprocessed intermediate image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size;

[0108] The second image determination subunit is used to preprocess and sample the first image to determine a second image with the intermediate image size.

[0109] Based on the above technical solutions, the intermediate image size determination subunit is specifically used for:

[0110] The target size is multiplied by the preset downsampling factor corresponding to the target downsampling network model, and the result of the multiplication is used as the size of the preprocessed intermediate image.

[0111] Based on the above technical solutions, the target downsampling network model includes: a pixel rearrangement sub-model and a convolution processing sub-model;

[0112] The downsampling processing module 740 includes:

[0113] The downsampling rearrangement unit is used to input the second image into the pixel rearrangement sub-model to perform downsampling rearrangement of image pixels to obtain a first feature map with the target size;

[0114] The convolution processing unit is used to input the first feature map into the convolution processing sub-model for feature convolution processing to obtain a target image with the target size.

[0115] Based on the above technical solutions, when the preset downsampling multiple corresponding to the target downsampling network model is an integer multiple, the pixel rearrangement sub-model includes: a first pixel inverse rearrangement module;

[0116] In this module, the channel amplification factor in the first pixel inverse rearrangement module is equal to the preset downsampling factor corresponding to the target downsampling network model.

[0117] Based on the above technical solutions, when the preset downsampling ratio of the target downsampling network model is a fractional ratio, the pixel rearrangement sub-model includes: a second pixel inverse rearrangement module, a first convolution module, and a pixel rearrangement module.

[0118] The channel amplification factor in the second pixel inverse rearrangement module and the channel reduction factor in the pixel rearrangement module are determined based on the preset downsampling factor corresponding to the target downsampling network model.

[0119] Based on the above technical solutions, the convolutional processing sub-model includes: a second convolutional module, a third convolutional module, and a fourth convolutional module;

[0120] The convolution processing unit is specifically used for: inputting the first feature map into the second convolution layer for convolution processing to obtain the processed second feature map; inputting the second feature map into the third convolution layer for convolution processing to obtain the processed third feature map; and inputting the second feature map and the third feature map into the fourth convolution layer for convolution processing to obtain a target image with the target size.

[0121] Based on the above technical solutions, the device also includes:

[0122] The downsampling network model training module is used to upsample the sample image by using a preset downsampling ratio corresponding to the downsampling network model as an upsampling ratio to obtain an upsampled image; input the upsampled image into the downsampling network model to be trained for downsampling processing to obtain the output image of the downsampling network model; determine the training error based on the output image and the sample image, and backpropagate the training error to the downsampling network model to be trained for network parameter adjustment until a preset convergence condition is reached, at which point the training of the downsampling network model is considered complete.

[0123] The image processing apparatus provided in this disclosure can execute the image processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the image processing method.

[0124] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0125] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 8 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 8The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0126] like Figure 8 As shown, electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0127] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0128] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0129] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0130] The electronic device provided in this embodiment and the image processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0131] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the image processing method provided in the above embodiments.

[0132] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0134] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0135] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine a target downsampling ratio corresponding to the first image based on the original size and the processed target size of the first image to be processed; determine a target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio corresponding to the downsampling network model; determine a second image that satisfies a preset downsampling condition based on the preset downsampling ratio corresponding to the target downsampling network model, the target downsampling ratio, and the first image; and perform downsampling processing on the second image based on the target downsampling network model to obtain a target image with the target size.

[0136] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0139] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0140] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] According to one or more embodiments of this disclosure, [Example 1] provides an image processing method, including:

[0142] Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image;

[0143] Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, the target downsampling network model corresponding to the first image is determined.

[0144] Based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image, a second image that meets the preset downsampling conditions is determined;

[0145] Based on the target downsampling network model, the second image is downsampled to obtain a target image with the target size.

[0146] According to one or more embodiments of this disclosure, [Example 2] provides an image processing method, further comprising:

[0147] Optionally, determining the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio corresponding to the downsampling network model includes:

[0148] Determine the ratio difference between the target downsampling ratio and the preset downsampling ratio corresponding to each pre-trained downsampling network model;

[0149] Based on the magnification difference corresponding to each downsampling network model, the target downsampling network model corresponding to the first image is determined.

[0150] According to one or more embodiments of this disclosure, [Example 3] provides an image processing method, further comprising:

[0151] Optionally, determining a second image that satisfies the preset downsampling conditions based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image includes:

[0152] If the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio, then the first image is determined as the second image that satisfies the preset downsampling condition;

[0153] If the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, then based on the preset downsampling ratio corresponding to the target downsampling network model and the target size, the first image is preprocessed and sampled to determine a second image that meets the preset downsampling conditions.

[0154] According to one or more embodiments of this disclosure, [Example 4] provides an image processing method, further comprising:

[0155] Optionally, the step of preprocessing and sampling the first image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size to determine the second image that meets the preset downsampling conditions includes:

[0156] Based on the preset downsampling ratio corresponding to the target downsampling network model and the target size, the size of the preprocessed intermediate image is determined;

[0157] The first image is preprocessed and sampled to determine a second image with the intermediate image size.

[0158] According to one or more embodiments of this disclosure, [Example 5] provides an image processing method, further comprising:

[0159] Optionally, determining the size of the preprocessed intermediate image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size includes:

[0160] The target size is multiplied by the preset downsampling factor corresponding to the target downsampling network model, and the result of the multiplication is used as the size of the preprocessed intermediate image.

[0161] According to one or more embodiments of this disclosure, [Example Six] provides an image processing method, further comprising:

[0162] Optionally, the target downsampling network model includes: a pixel rearrangement sub-model and a convolutional processing sub-model;

[0163] The step of downsampling the second image based on the target downsampling network model to obtain a target image with the target size includes:

[0164] The second image is input into the pixel rearrangement sub-model to perform downsampling and rearrangement of image pixels, thereby obtaining a first feature map with the target size;

[0165] The first feature map is input into the convolution processing sub-model for feature convolution processing to obtain a target image with the target size.

[0166] According to one or more embodiments of this disclosure, [Example Seven] provides an image processing method, further comprising:

[0167] Optionally, when the preset downsampling factor corresponding to the target downsampling network model is an integer multiple, the pixel rearrangement sub-model includes: a first pixel inverse rearrangement module;

[0168] In this module, the channel amplification factor in the first pixel inverse rearrangement module is equal to the preset downsampling factor corresponding to the target downsampling network model.

[0169] According to one or more embodiments of this disclosure, [Example Eight] provides an image processing method, further comprising:

[0170] Optionally, when the preset downsampling factor corresponding to the target downsampling network model is a fractional factor, the pixel rearrangement sub-model includes: a second pixel inverse rearrangement module, a first convolution module, and a pixel rearrangement module;

[0171] The channel amplification factor in the second pixel inverse rearrangement module and the channel reduction factor in the pixel rearrangement module are determined based on the preset downsampling factor corresponding to the target downsampling network model.

[0172] According to one or more embodiments of this disclosure, [Example Nine] provides an image processing method, further comprising:

[0173] Optionally, the convolutional processing sub-model includes: a second convolutional module, a third convolutional module, and a fourth convolutional module;

[0174] The step of inputting the first feature map into the convolution processing sub-model for feature convolution processing to obtain a target image with the target size includes:

[0175] The first feature map is input into the second convolutional layer for convolution processing to obtain the processed second feature map;

[0176] The second feature map is input into the third convolutional layer for convolution processing to obtain the processed third feature map;

[0177] The second feature map and the third feature map are input into the fourth convolutional layer for convolution processing to obtain a target image with the target size.

[0178] According to one or more embodiments of this disclosure, [Example 10] provides an image processing method, further comprising:

[0179] Optionally, the training process for each downsampled network model includes:

[0180] The preset downsampling factor corresponding to the downsampling network model is used as the upsampling factor to perform upsampling processing on the sample image to obtain the upsampled image;

[0181] The upsampled image is input into the downsampled network model to be trained for downsampling processing to obtain the output image of the downsampled network model;

[0182] The training error is determined based on the output image and the sample image, and the training error is backpropagated to the downsampled network model to be trained for network parameter adjustment until the preset convergence condition is reached, at which point the training of the downsampled network model is considered complete.

[0183] According to one or more embodiments of this disclosure, [Example 11] provides an image processing apparatus, including:

[0184] The downsampling ratio determination module is used to determine the target downsampling ratio of the first image based on the original size of the first image to be processed and the target size after processing.

[0185] The network model determination module is used to determine the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model.

[0186] The second image determination module is used to determine a second image that meets the preset downsampling conditions based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image.

[0187] The downsampling processing module is used to perform downsampling processing on the second image based on the target downsampling network model to obtain a target image with the target size.

[0188] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0189] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0190] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, characterized in that, include: Based on the original size of the first image to be processed and the target size after processing, determine the target downsampling factor corresponding to the first image; Based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model, the target downsampling network model corresponding to the first image is determined. Based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image, a second image that meets the preset downsampling conditions is determined; Based on the target downsampling network model, the second image is downsampled to obtain a target image with the target size. The target downsampling network model includes: a pixel rearrangement sub-model and a convolutional processing sub-model; When the preset downsampling factor corresponding to the target downsampling network model is an integer multiple, the pixel rearrangement sub-model includes: a first pixel inverse rearrangement module; When the preset downsampling factor corresponding to the target downsampling network model is a fractional factor, the pixel rearrangement sub-model includes: a second pixel inverse rearrangement module, a first convolution module, and a pixel rearrangement module.

2. The image processing method according to claim 1, characterized in that, The step of determining the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and a preset downsampling ratio corresponding to the downsampling network model includes: Determine the ratio difference between the target downsampling ratio and the preset downsampling ratio corresponding to each pre-trained downsampling network model; Based on the magnification difference corresponding to each downsampling network model, the target downsampling network model corresponding to the first image is determined.

3. The image processing method according to claim 1, characterized in that, The step of determining a second image that meets the preset downsampling conditions based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image includes: If the preset downsampling ratio corresponding to the target downsampling network model is equal to the target downsampling ratio, then the first image is determined as the second image that satisfies the preset downsampling condition; If the preset downsampling ratio corresponding to the target downsampling network model is not equal to the target downsampling ratio, then based on the preset downsampling ratio corresponding to the target downsampling network model and the target size, the first image is preprocessed and sampled to determine a second image that meets the preset downsampling conditions.

4. The image processing method according to claim 3, characterized in that, The step of preprocessing and sampling the first image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size to determine the second image that meets the preset downsampling conditions includes: Based on the preset downsampling ratio corresponding to the target downsampling network model and the target size, the size of the preprocessed intermediate image is determined; The first image is preprocessed and sampled to determine a second image with the intermediate image size.

5. The image processing method according to claim 4, characterized in that, The step of determining the size of the preprocessed intermediate image based on the preset downsampling ratio corresponding to the target downsampling network model and the target size includes: The target size is multiplied by the preset downsampling factor corresponding to the target downsampling network model, and the result of the multiplication is used as the size of the preprocessed intermediate image.

6. The image processing method according to claim 1, characterized in that, The step of downsampling the second image based on the target downsampling network model to obtain a target image with the target size includes: The second image is input into the pixel rearrangement sub-model to perform downsampling and rearrangement of image pixels, thereby obtaining a first feature map with the target size; The first feature map is input into the convolution processing sub-model for feature convolution processing to obtain a target image with the target size.

7. The image processing method according to claim 6, characterized in that, The channel amplification factor in the first pixel inverse rearrangement module is equal to the preset downsampling factor corresponding to the target downsampling network model.

8. The image processing method according to claim 6, characterized in that, The channel amplification factor in the second pixel inverse rearrangement module and the channel reduction factor in the pixel rearrangement module are determined based on the preset downsampling factor corresponding to the target downsampling network model.

9. The image processing method according to claim 6, characterized in that, The convolutional processing sub-model includes: a second convolutional module, a third convolutional module, and a fourth convolutional module; The step of inputting the first feature map into the convolution processing sub-model for feature convolution processing to obtain a target image with the target size includes: The first feature map is input into the second convolutional layer for convolution processing to obtain the processed second feature map; The second feature map is input into the third convolutional layer for convolution processing to obtain the processed third feature map; The second feature map and the third feature map are input into the fourth convolutional layer for convolution processing to obtain a target image with the target size.

10. The image processing method according to any one of claims 1-9, characterized in that, The training process for each downsampling network model includes: The preset downsampling factor corresponding to the downsampling network model is used as the upsampling factor to perform upsampling processing on the sample image to obtain the upsampled image; The upsampled image is input into the downsampled network model to be trained for downsampling processing to obtain the output image of the downsampled network model; The training error is determined based on the output image and the sample image, and the training error is backpropagated to the downsampled network model to be trained for network parameter adjustment until the preset convergence condition is reached, at which point the training of the downsampled network model is considered complete.

11. An image processing apparatus, characterized in that, include: The downsampling ratio determination module is used to determine the target downsampling ratio of the first image based on the original size of the first image to be processed and the target size after processing. The network model determination module is used to determine the target downsampling network model corresponding to the first image based on the target downsampling ratio, at least one pre-trained downsampling network model, and the preset downsampling ratio corresponding to the downsampling network model. The second image determination module is used to determine a second image that meets the preset downsampling conditions based on the preset downsampling factor corresponding to the target downsampling network model, the target downsampling factor, and the first image. The downsampling processing module is used to perform downsampling processing on the second image based on the target downsampling network model to obtain a target image with the target size; The target downsampling network model includes: a pixel rearrangement sub-model and a convolutional processing sub-model; When the preset downsampling factor corresponding to the target downsampling network model is an integer multiple, the pixel rearrangement sub-model includes: a first pixel inverse rearrangement module; When the preset downsampling factor corresponding to the target downsampling network model is a fractional factor, the pixel rearrangement sub-model includes: a second pixel inverse rearrangement module, a first convolution module, and a pixel rearrangement module.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the image processing method as described in any one of claims 1-10.

Images