Super-resolution image generation method, device and storage medium

Abstract

The present disclosure provides a super-resolution image generation method, device and storage medium, the method comprising: obtaining a three-channel original image and a target resolution CxD, wherein the target resolution CxD is greater than the original resolution AxE of the original image, and converting the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image and two green channel images, and processing the four single-channel images according to the target resolution CxD to obtain four single-channel images with a resolution of CxD, combining the four single-channel images with a resolution of CxD to obtain a RAW format image with a resolution of CxD, and processing the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of CxD, which can improve the super-resolution processing effect of a single image compared with traditional methods, and does not require model training, thus also reducing the computational load.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a super-resolution image generation method, device, and storage medium. Background Art

[0002] Super-resolution algorithms improve the resolution of existing images through hardware or software methods. In related technologies, image super-resolution algorithms are primarily categorized into two types: traditional and deep learning. Traditional super-resolution algorithms include interpolation, sparse representation, local embedding, and example-based methods. Deep learning methods use neural network models to reconstruct image resolution.

[0003] However, most traditional methods suffer from performance drawbacks. For example, super-resolved images generated by interpolation methods are often blurry; super-resolved images generated by sparse representations exhibit noticeable jagged edges; local embedding relies on the dataset, making it difficult to apply to a single image; and example-based methods require very high levels of detail and texture within a single image, resulting in poor results for low-resolution images without much repetitive detail. Deep learning methods, on the other hand, are computationally intensive and can generate artifacts that are not present in the original image, compromising the quality of super-resolution images. Summary of the Invention

[0004] The present disclosure proposes a super-resolution image generation method, device, and storage medium, aiming to solve one of the technical problems in the related art at least to a certain extent.

[0005] The first embodiment of the present disclosure proposes a super-resolution image generation method, comprising: obtaining a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than the original resolution A×E of the original image; converting the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images; processing the four single-channel images according to the target resolution C×D to obtain a resolution of Four single-channel images with resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D; and the RAW format image is processed by using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D.

[0006] The second embodiment of the present disclosure proposes a super-resolution image generation device, comprising: a first acquisition module, configured to acquire a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than the original resolution A×E of the original image; a conversion module, configured to convert the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images; a first processing module, configured to process the four single-channel images according to the target resolution C×D to obtain a super-resolution image with a resolution of Four single-channel images with a resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D; and a second processing module is used to process the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D.

[0007] The third aspect embodiment of the present disclosure proposes a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the super-resolution image generation method of the embodiment of the present disclosure.

[0008] The fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the super-resolution image generation method disclosed in the embodiment of the present disclosure.

[0009] In this embodiment, a three-channel original image and a target resolution C×D are obtained, wherein the target resolution C×D is greater than the original resolution A×E of the original image, and the original image is converted into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images, and the four single-channel images are processed according to the target resolution C×D to obtain a resolution of Four single-channel images with a resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, and the RAW format image is processed by the demosaicing algorithm to obtain a three-channel target image with a resolution of C×D. Compared with the traditional method, it can improve the super-resolution processing effect of a single image, and does not require model training, so it can also reduce the amount of calculation.

[0010] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0012] Figure 1 1 is a flow chart of a super-resolution image generation method according to an embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram of a RAW image of a Bayer array provided according to an embodiment of the present disclosure;

[0014] Figure 3a This is a schematic diagram of a color structure of pixels within a processing window provided according to an embodiment of the present disclosure;

[0015] Figure 3b is a schematic diagram of another color structure of pixels within a processing window provided according to an embodiment of the present disclosure;

[0016] Figure 4 is a flowchart of a super-resolution image generation method provided according to another embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of single-channel image interpolation provided according to an embodiment of the present disclosure;

[0018] Figure 6 is a schematic diagram of the super-resolution image processing process provided by an embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram of a super-resolution image generating apparatus provided according to another embodiment of the present disclosure;

[0020] Figure 8 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present disclosure and are not to be construed as limiting the present disclosure. On the contrary, the embodiments of the present disclosure include all variations, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0022] It should be noted that the executor of the super-resolution image generation method of this embodiment may be a super-resolution image generation device, which may be implemented by software and / or hardware, and which may be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0023] Figure 1 FIG. 1 is a flow chart of a super-resolution image generation method according to an embodiment of the present disclosure, Figure 1 As shown, the method includes:

[0024] S101: Acquire three-channel original images and target resolution C×D.

[0025] The single image that needs to be super-resolution processed can be called the original image, which can be represented by Image_In. In this embodiment, the original image is a three-channel image, i.e., an RGB image, and the resolution of the original image can be called the original resolution, which can be represented by A×E, for example.

[0026] The target resolution, which indicates the image resolution after super-resolution processing, can be expressed as C×D. In this embodiment, the target resolution C×D is greater than the original resolution A×E of the original image, that is, A is smaller than C, and E is smaller than D. Therefore, in this embodiment, the original image with a low resolution of A×E is super-resolution processed to obtain a target image with a high resolution of C×D.

[0027] In some embodiments, when super-resolution processing is required for an image, the user may input an original image with a resolution of A×E and a target resolution of C×D in a preset client. In this case, the present embodiment may obtain a three-channel original image and a target resolution of C×D.

[0028] S102: Converting the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images.

[0029] That is to say, this embodiment can separate the original RGB three-channel image Image_In to obtain multiple single-channel images. In some embodiments, the multiple single-channel images are, for example, four single-channel images, including a red channel image (R_In), a blue channel image (B_In), and two green channel images (G_In1 and G_In2).

[0030] Specifically, this embodiment can separate the original image Image_In into three single-channel images, namely, a red channel image R_In, a blue channel image B_In, and a green channel image G_In, and copy the green channel image G_In separately to obtain two green channel images G_In1 and G_In2.

[0031] It can be understood that the resolution of the separated single-channel image is the same as the original resolution A×E of the original image.

[0032] S103: Process the four single-channel images according to the target resolution C×D to obtain a resolution of Four single-channel images of .

[0033] After obtaining the four single-channel images, the present embodiment can further process the four single-channel images according to the target resolution C×D to obtain four images with a resolution of A single-channel image, which can be expressed as R_Out, G_Out1, B_Out, G_Out2.

[0034] In some embodiments, for example, an interpolation or sampling operation may be performed on each single-channel image with a resolution of A×E to obtain a single-channel image with a resolution of A single-channel image.

[0035] S104: Set the resolution to The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, which can be represented by Raw_1.

[0036] Furthermore, in this embodiment, the above four resolutions are The single-channel images R_Out, G_Out1, B_Out, and G_Out2 are combined to obtain a RAW format image with a resolution of C × D. There is no restriction on the combination method.

[0037] Some embodiments, Figure 2 is a schematic diagram of a RAW image of a Bayer array provided according to an embodiment of the present disclosure, such as Figure 2 As shown, in this embodiment, for example, four single-channel images R_Out, G_Out1, B_Out, and G_Out2 can be combined according to a Bayer pattern to obtain a RAW format image.

[0038] S105: Process the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D.

[0039] In practical applications, directly interpolating four single-channel images with a resolution of C×D and then combining them into an EGB format image does not introduce enough nonlinearity, causing the final image to become very blurred.

[0040] In view of this, this embodiment may use a demosaicing algorithm to interpolate the RAW format image into a three-channel target image with a resolution of C×D, which may be represented by Image_Out.

[0041] In some embodiments, for example, an adaptive demosaicing algorithm may be used to interpolate the RAW format image. Figure 3ais a schematic diagram of a color structure of pixels in a processing window provided according to an embodiment of the present disclosure, such as Figure 3a As shown, the processing window of the adaptive demosaicing algorithm in this embodiment can be 5*5.

[0042] Specifically, during the processing, this embodiment can first loop through each pixel point in the 5*5 processing window of the RAW format image to obtain the currently traversed target pixel point Pixel, and further perform color interpolation on the target pixel point according to the color component of the target pixel point Pixel (i.e., red, blue or green).

[0043] In some embodiments, when the target pixel point is a red / blue component, a first interpolation direction is calculated, and a green component is inserted into the target pixel point according to the first interpolation direction, that is, when the current target pixel point Pixel is a red pixel point R or a blue pixel point B, the green component G is inserted.

[0044] Specifically, if Figure 3a As shown, this example first calculates the horizontal gradient Grad H and vertical gradient Grad V ,in,

[0045] Grad H =abs(R 22 -R 20 )+abs(R 22 -R 24 )+abs(R 20 -G 21 )+abs(G 21 -R 22 )+abs(R 22 -G 23 )+abs(G 23 -R 24 )

[0046] Grad V =abs(R 22 -R 02 )+abs(R 22 -R 42 )+abs(R 02 -G 12 )+abs(G12-R22)+abs(R22-G 32 )+abs(G 32 -R 42 )

[0047] Furthermore, the horizontal gradient Grad is selected H and vertical gradient Grad VThe direction of the smaller gradient is used as the first interpolation direction, and color ratio interpolation is performed.

[0048] For example, the target pixel is R 22 , if the minimum gradient direction is horizontal (i.e., the first interpolation direction), then for the target pixel R 22 The inserted green component can be expressed as If the minimum gradient direction is the vertical direction (i.e., the first interpolation direction), the interpolated green component can be expressed as It should be noted that if the interpolation direction is horizontal, the final interpolation Interp G Need to be limited to [min(G 21 , G 23 ), max(G 21 , G 23 )]; If the interpolation direction is vertical, the final interpolation Interp G Need to be limited to [min(G 12 , G 32 ), max(G 12 , G 32 )].

[0049] In some other embodiments, during the second round of traversal, if the target pixel has a red / blue component, a second interpolation direction is calculated, and the blue / red component is interpolated into the target pixel according to the second interpolation direction. That is, if the traversed pixel is a red pixel, the blue component B is interpolated; if the traversed pixel is a blue pixel, the red component R is interpolated. This embodiment uses the example of inserting the blue component B when the target pixel is an R pixel as an example for illustrative description. The processing window size is 5x5, and the current pixel is the R channel. During the second round of traversal, the B pixels at positions 11, 13, 31, and 33 already have the green component G. The method for interpolating the red component R into the B pixel is similar.

[0050] Specifically, if Figure 3a As shown, this example first calculates the 45-degree directional gradient Grad 45 and 135 degree directional gradient Grad 135 ,in,

[0051] Grad 45 =abs(R 22 -R 00 )+abs(R 22 -R 44 )+abs(R 00 -G 11 )+abs(G 11 -R 22 )+abs(R 22 G 33)+abs(G 33 -R 44 )

[0052] Grad 135 =abs(R 22 -R 04 )+abs(R 22 -R 40 )+abs(R 04 -G 13 )+abs(G 13 -R 22 )+abs(R 22 -G 31 )+abs(G 31 R 40 )

[0053] Furthermore, select the 45-degree directional gradient Grad 45 and 135 degree directional gradient Grad 135 The direction of the smaller gradient is used as the second interpolation direction, and color ratio interpolation is performed.

[0054] For example, if the minimum gradient direction is 45 degrees (ie, the second interpolation direction), then for the target pixel R 22 The inserted blue component can be expressed as If the minimum gradient direction is 135 degrees (i.e., the second interpolation direction), the blue component inserted into the target pixel R22 can be expressed as It should be noted that if the interpolation direction is 45 degrees, the final interpolation Interp B Need to be limited to [min(B 11 , B 33 ), max(B 11 , B 33 )]; If the interpolation direction is 135 degrees, the final interpolation Interp B Need to be limited to [min(B 13 , B 31 ), max(B 13 , B 31 )].

[0055] In some other embodiments, during the third round of traversal, as Figure 3b As shown, when the target pixel is a green component, the third interpolation direction is calculated, and the red / blue components are interpolated to the target pixel according to the third interpolation direction. That is to say, when the target pixel is a green pixel, in this case, the red component R and the blue component B are interpolated. This embodiment takes the current target pixel as a green pixel G and the red component is inserted as an example. The processing window size is 5x5. It should be noted that in the third round of traversal, in addition to Figure 3b The G pixel, R and B pixel points are all full channels.

[0056] Specifically, if Figure 3b As shown, this example first calculates the horizontal gradient Grad H and vertical gradient Grad V ,in,

[0057] Grad H =abs(G 22 -G 20 )+abs(G 22 -G 24 )+abs(G 20 -G 21 )+abs(G 21 -G 22 )+abs(G 22 -G 23 )+abs(G 23 -G 24 )

[0058] Grad V =abs(G 22 -G 02 )+abs(G 22 -G 42 )+abs(G 02 -G 12 )+abs(G 12 -G 22 )+abs(G 22 -G 32 )+abs(G 32 -G 42 )

[0059] Furthermore, the horizontal gradient Grad is selected H and vertical gradient Grad V The direction of the smaller gradient is used as the third interpolation direction for color ratio interpolation. In this embodiment, G interpolation to R is taken as an example, and G interpolation to B is similar.

[0060] For example, if the minimum gradient direction is the horizontal direction (ie, the third interpolation direction), then for the target pixel point R 22 The inserted red component can be expressed as If the minimum gradient direction is the vertical direction (i.e., the third interpolation direction), the red component inserted into the target pixel R22 can be expressed as It should be noted that the interpolation direction is horizontal, so the final interpolation Interp R Need to be limited to [min(R 21 , R23 ), max(R 21 , R 23 )]; If the interpolation direction is vertical, the final interpolation Interp R Need to be limited to [min(R 12 , R 32 ), max(R 12 , R 32 )].

[0061] Thus, after multiple rounds of interpolation operations, the final RGB three-channel image is obtained as the target image with a resolution of C×D, that is, a super-resolution image.

[0062] Compared with the above-mentioned traditional method, this embodiment uses a demosaicing algorithm to improve the shortcomings of easy introduction of aliasing and insufficient clarity, and uses color ratio interpolation to make the final result more delicate and sharper.

[0063] In this embodiment, a three-channel original image and a target resolution C×D are obtained, wherein the target resolution C×D is greater than the original resolution A×E of the original image, and the original image is converted into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images, and the four single-channel images are processed according to the target resolution C×D to obtain a resolution of Four single-channel images with a resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, and the RAW format image is processed by the demosaicing algorithm to obtain a three-channel target image with a resolution of C×D. Compared with the traditional method, it can improve the super-resolution processing effect of a single image, and does not require model training, so it can also reduce the amount of calculation.

[0064] Figure 4 FIG. 1 is a flow chart of a super-resolution image generation method according to an embodiment of the present disclosure, Figure 4 As shown, the method includes:

[0065] S401: Acquire three-channel original images, target resolution C×D, and target clarity.

[0066] In the disclosed embodiment, when the user inputs the original resolution A×E and the target resolution C×D, the user may also input a set target definition, which may be represented by α. In this case, the embodiment may obtain an original image with a resolution of A×E, a target resolution C×D, and a target definition α.

[0067] S402: Convert the original image into four single-channel images, where the four single-channel images include a red channel image, a blue channel image, and two green channel images.

[0068] The specific description of S402 is given in the above embodiment and will not be repeated here.

[0069] S403: Calculate a horizontal sampling interpolation rate and a vertical sampling interpolation rate according to the target resolution C×D and the original resolution A×E.

[0070] In this embodiment, the interpolation processing of four single-channel images is mainly divided into horizontal interpolation and vertical interpolation. Specifically, first, according to the target resolution C×D and the original resolution A×E, the horizontal sampling interpolation rate (H_rate) and the vertical sampling interpolation rate (V_rate) are calculated, where:

[0071] S404: According to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition, each single-channel image is interpolated to obtain a resolution of Four single-channel images of .

[0072] That is to say, according to the horizontal sampling interpolation rate H_rate, the vertical sampling interpolation rate V_rate and the target clarity α, the four single-channel images R_In, B_In, G_In1, and G_In2 are interpolated to obtain a resolution of The interpolation process for each single channel is the same, and the following example will take the processing of R_In as an example for illustration.

[0073] In some embodiments, the first row, first column, last row, and last column of a single-channel image may constitute the edge area of ​​the image, and the remaining areas of the image except the edge area may be referred to as other areas. In this embodiment, interpolation operations may be performed on the pixel points of the edge area and other areas respectively.

[0074] Specifically, during the interpolation of the edge region, this embodiment can interpolate the pixels in the first row, first column, last row, and last column of each single-channel image based on H_rate, V_rate, and the target definition α. ​​Specifically, the first and last rows of pixels of the single-channel image can be interpolated based on the horizontal sampling interpolation rate H_rate and the target definition α; similarly, the first and last columns of pixels of the single-channel image can be interpolated based on the vertical sampling interpolation rate V_rate and the target definition α.

[0075] In some embodiments, multiple first positions to be interpolated in the first and last rows may be determined based on the horizontal sampling interpolation rate H_rate, and the first pixel value of each first position to be interpolated may be calculated based on the target definition and the pixel values ​​of two pixels adjacent to each first position to be interpolated. The interpolation process for the first and last rows is the same, and this embodiment will use the first row interpolation as an example for illustration.

[0076] The position in the first row that needs to be interpolated can be referred to as a first position to be interpolated, and the pixel value of the first position to be interpolated can be referred to as a first pixel value.

[0077] Specifically, for the first row, its horizontal sampling interpolation rate H_rate is simplified to a fractional form without a common divisor, which can be expressed as That is, if every k pixels in the first row need to be expanded to f pixels, then fk pixels can be obtained by interpolating k pixels. In some embodiments, the position between two adjacent pixels can be used as the first position to be interpolated, and then fk pixels can be evenly interpolated to the first position to be interpolated.

[0078] For example, Figure 5 is a schematic diagram of a single-channel image interpolation according to an embodiment of the present disclosure, such as Figure 5 As shown, for example, f=5, k=3, white represents the edge area before interpolation, and gray represents the edge area after interpolation. That is to say, every 3 pixels in the first row of the single-channel image (from left to right, for example, pixel 1, pixel 3, pixel 5) can form two first positions to be interpolated, and then 2 pixels (pixel 2 and pixel 5) can be inserted.

[0079] Furthermore, a first pixel value is calculated according to the target definition α and the pixel values ​​of two pixel points adjacent to each first position to be interpolated.

[0080] For example, the first position to be interpolated is the position between pixel 1 and pixel 3, and the two pixel points adjacent to the first position to be interpolated are pixel 1 and pixel 3. In this case, the first pixel value of the first position to be interpolated, that is, the pixel value of pixel 2, can be calculated based on the pixel values ​​of pixel 1 and pixel 3 and the target clarity α. The pixel values ​​of pixel 1 to pixel 5 can be expressed as Pixel 1, Pixel 2, Pixel 3, Pixel 4, and Pixel 5, for example. In some embodiments, the pixel value of pixel 2 (that is, the first pixel value) can be expressed as Similarly, the pixel value Pixel4 (i.e., the first pixel value) of pixel 4 can be expressed as α is usually set to be greater than 0.5 so that the newly interpolated pixel is closer to the middle value. It is understandable that the number of pixels to be interpolated at the first position to be interpolated can be calculated based on k and fk.

[0081] Similarly, in this embodiment, multiple second positions to be interpolated in the first column and the last column can be determined according to the vertical sampling interpolation rate V_rate, and the second pixel value of the second position to be interpolated can be calculated according to the target clarity α and the pixel values ​​of two pixels adjacent to each second position to be interpolated. The difference between the first column and the last column is similar to the horizontal interpolation described above and will not be repeated here.

[0082] Therefore, this embodiment uses a non-uniform transition method to perform interpolation operations on the first row and first column and the last row and last column, which can effectively reduce the transition band and prevent the excessive loss of clarity in traditional bilinear interpolation, making the final image less blurred.

[0083] It should be noted that, for the case where f and k are other values ​​and interpolation in the middle is impossible, the main principle of interpolation for the first row and first column and the last row and last column in this embodiment is that the new interpolation points can be concentrated on the broken line composed of the original pixel points to increase the overall variance.

[0084] However, this embodiment has a significant impact on other areas (such as Figure 5 During the interpolation process of the gray pixel in the center of the image, the bicubic interpolation method can be used to interpolate other areas of the single-channel image according to the horizontal sampling interpolation rate, vertical sampling interpolation rate and target clarity, and the block with a side length of k is extended to the block with a side length of f. Thus, each single-channel image can be expanded from a resolution of A×E to

[0085] S405: Set the resolution to The four single-channel images are combined to obtain a RAW format image with a resolution of C×D.

[0086] S406: Process the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D.

[0087] The specific description of S405-406 can be found in the above embodiment and will not be repeated here.

[0088] In this embodiment, a three-channel original image and a target resolution C×D are obtained, wherein the target resolution C×D is greater than the original resolution A×E of the original image, and the original image is converted into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images, and the four single-channel images are processed according to the target resolution C×D to obtain a resolution of Four single-channel images with a resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, and the RAW format image is processed using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D. Compared with traditional methods, this can improve the super-resolution processing effect of a single image and does not require model training, thereby reducing the amount of calculation. In addition, this embodiment uses a non-uniform transition method to interpolate the first row and first column and the last row and last column, which can effectively reduce the transition band and prevent the excessive loss of clarity caused by traditional bilinear interpolation, making the final image less blurry.

[0089] In one specific example, Figure 6 is a schematic diagram of the super-resolution image processing process provided by the embodiment of the present disclosure, such as Figure 6 As shown, the following steps are included:

[0090] Step 1: Input image Image_In with resolution A x E, definition α and target resolution C x D;

[0091] Step 2: Separate the three channels of the input image Image_In to obtain three single-channel input images R_In, G_In and B_In;

[0092] Step 3: Duplicate the G channel image. Now there are four single channel input images R_In, G_In1, G_In2 and B_In;

[0093] Step 4: Calculate the horizontal and vertical sampling interpolation rates H_rate and V_rate according to the input image resolution A x E and the target resolution C x D;

[0094] Step 5: Perform sampling interpolation operations on the four single-channel input images according to the sampling interpolation rates H_rate, V_rate and the resolution α. ​​Obtain four single-channel output images R_Out, G_Out1, G_Out2, B_Out with a resolution of (C / 2)x(D / 2);

[0095] Step 6: Combine the four single-channel output images R_Out, G_Out1, G_Out2, and B_Out into a RAW image (original image) Raw_1 with a resolution of C x D according to the Bayer Pattern;

[0096] Step 7: Loop through each pixel of Raw_1 and calculate the optimal interpolation direction when the current point is R / B. Interpolate the G pixel according to the interpolation direction;

[0097] Step 8: When the current point is R / B, calculate the optimal interpolation direction. Interpolate the B / R pixel point according to the interpolation direction;

[0098] Step 9: When the current point is G, calculate the optimal interpolation direction. Interpolate B / R pixels according to the interpolation direction;

[0099] Step 10: Get the complete RGB three-channel image Image_Out with a resolution of C x D. Image_Out is the final super-resolution image.

[0100] Figure 7 FIG. 1 is a schematic diagram of a super-resolution image generation device according to another embodiment of the present disclosure. Figure 7 As shown, the super-resolution image generating device 70 includes:

[0101] A first acquisition module 701 is configured to acquire a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than an original resolution A×E of the original image;

[0102] A conversion module 702 is configured to convert the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images;

[0103] The first processing module 703 is used to process the four single-channel images according to the target resolution C×D to obtain a resolution of Four single-channel images of;

[0104] Combining module 704, for converting the resolution Combine the four single-channel images to obtain a RAW format image with a resolution of C×D; and

[0105] The second processing module 705 is configured to process the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D.

[0106] In some embodiments, the apparatus 70 further includes: a second acquisition module, configured to acquire target clarity;

[0107] Furthermore, the first processing module includes:

[0108] a calculation submodule, configured to calculate a horizontal sampling interpolation rate and a vertical sampling interpolation rate according to a target resolution C×D and an original resolution A×E; and

[0109] The interpolation submodule is used to interpolate each single-channel image according to the horizontal sampling interpolation rate, vertical sampling interpolation rate and target clarity to obtain a resolution of Four single-channel images of .

[0110] In some embodiments, the calculation submodule is specifically used to: interpolate the edge area of ​​each single-channel image according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target clarity, wherein the edge area includes the first row, the first column, the last row and the last column; and interpolate other areas of the single-channel image using the bicubic interpolation method according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target clarity.

[0111] In some embodiments, the calculation submodule is specifically used to: determine multiple first positions to be interpolated in the first row and the last row according to the horizontal sampling interpolation rate, and calculate the first pixel value of the first position to be interpolated according to the target clarity and the pixel values ​​of two pixel points adjacent to each first position to be interpolated; and determine multiple second positions to be interpolated in the first column and the last column according to the vertical sampling interpolation rate, and calculate the second pixel value of the second position to be interpolated according to the target clarity and the pixel values ​​of two pixel points adjacent to each second position to be interpolated.

[0112] In some embodiments, the second processing module 705 is specifically used to: loop through the pixel points of the RAW format image to obtain the target pixel point; when the target pixel point is a red / blue component, calculate the first interpolation direction, and insert the green component into the target pixel point according to the first interpolation direction; when the target pixel point is a red / blue component, calculate the second interpolation direction, and insert the blue / red component into the target pixel point according to the second interpolation direction; when the target pixel point is a green component, calculate the third interpolation direction, and insert the red / blue component into the target pixel point according to the third interpolation direction.

[0113] In some embodiments, the combination module is specifically configured to: convert the resolution The four single-channel images are combined according to the Bayer array to obtain a RAW format image with a resolution of C×D.

[0114] In this embodiment, by obtaining a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than the original resolution A×E of the original image, and converting the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images, and processing the four single-channel images according to the target resolution C×D, a resolution of Four single-channel images with a resolution of The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, and the RAW format image is processed by the demosaicing algorithm to obtain a three-channel target image with a resolution of C×D. Compared with the traditional method, it can improve the super-resolution processing effect of a single image, and does not require model training, so it can also reduce the amount of calculation.

[0115] According to an embodiment of the present disclosure, the present disclosure further provides a computer device, a readable storage medium, and a computer program product.

[0116] In order to implement the above embodiments, the present disclosure further proposes a computer program product. When an instruction processor in the computer program product executes, the super-resolution image generation method proposed in the above embodiments of the present disclosure is executed.

[0117] Figure 8 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 8 The computer device 12 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0118] like Figure 8 As shown, computer device 12 is implemented as a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0119] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0120] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0121] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 8 Not shown, often called a "hard drive").

[0122] although Figure 8 Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), a Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0123] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0124] The computer device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via an input / output (I / O) interface 22. Furthermore, the computer device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0125] The processing unit 16 executes various functional applications by running programs stored in the system memory 28 , such as implementing the super-resolution image generation method mentioned in the above embodiments.

[0126] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0127] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0128] It should be noted that, in the description of this disclosure, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0129] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0130] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0131] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0132] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0133] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0134] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0135] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are illustrative and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A super-resolution image generation method, characterized in that: include: Acquire a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than an original resolution A×E of the original image; Converting the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images; The four single-channel images are processed according to the target resolution C×D to obtain a resolution of Four single-channel images of; Set the resolution to Combine the four single-channel images to obtain a RAW format image with a resolution of C×D; and The RAW format image is processed using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D; wherein, the pixels of the RAW format image are traversed in a loop to obtain a target pixel; when the target pixel has a red / blue component, a first interpolation direction is calculated, and a green component is interpolated into the target pixel according to the first interpolation direction; when the target pixel has a red / blue component, a second interpolation direction is calculated, and a blue / red component is interpolated into the target pixel according to the second interpolation direction; when the target pixel has a green component, a third interpolation direction is calculated, and a red / blue component is interpolated into the target pixel according to the third interpolation direction.

2. The method according to claim 1, wherein The method further comprises: Gain clarity of purpose; Furthermore, processing the four single-channel images according to the target resolution C×D includes: Calculating a horizontal sampling interpolation rate and a vertical sampling interpolation rate according to the target resolution C×D and the original resolution A×E; and According to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition, each of the single-channel images is interpolated to obtain a resolution of Four single-channel images of .

3. The method according to claim 2, wherein The interpolation processing is performed on each of the single-channel images according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate, and the target definition, including: performing interpolation processing on the edge region of each single-channel image according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate, and the target definition, wherein the edge region includes the first row, the first column, the last row, and the last column; and According to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition, a bicubic interpolation method is used to perform interpolation processing on other areas of the single-channel image.

4. The method according to claim 3, wherein The interpolation processing is performed on the edge area of ​​each single-channel image according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition, comprising: determining a plurality of first positions to be interpolated in the first row and the last row according to the horizontal sampling interpolation rate, and calculating a first pixel value of each first position to be interpolated according to the target definition and pixel values ​​of two pixels adjacent to each first position to be interpolated; and A plurality of second positions to be interpolated in the first column and the last column are determined according to the vertical sampling interpolation rate, and a second pixel value of each second position to be interpolated is calculated according to the target definition and the pixel values ​​of two pixels adjacent to each second position to be interpolated.

5. The method according to any one of claims 1 to 4, characterized in that The resolution is The four single-channel images are combined to obtain a RAW format image with a resolution of C×D, including: Set the resolution to The four single-channel images are combined according to the Bayer array to obtain a RAW format image with a resolution of C×D.

6. A super-resolution image generation device, characterized in that: include: A first acquisition module is configured to acquire a three-channel original image and a target resolution C×D, wherein the target resolution C×D is greater than an original resolution A×E of the original image; a conversion module, configured to convert the original image into four single-channel images, wherein the four single-channel images include a red channel image, a blue channel image, and two green channel images; The first processing module is used to process the four single-channel images according to the target resolution C×D to obtain a resolution of Four single-channel images of; Combined module for converting resolution to Combine the four single-channel images to obtain a RAW format image with a resolution of C×D; and The second processing module is used to process the RAW format image using a demosaicing algorithm to obtain a three-channel target image with a resolution of C×D; wherein, the pixels of the RAW format image are traversed in a loop to obtain the target pixel; when the target pixel is a red / blue component, a first interpolation direction is calculated, and a green component is inserted into the target pixel according to the first interpolation direction; when the target pixel is a red / blue component, a second interpolation direction is calculated, and a blue / red component is inserted into the target pixel according to the second interpolation direction; when the target pixel is a green component, a third interpolation direction is calculated, and a red / blue component is inserted into the target pixel according to the third interpolation direction.

7. The device according to claim 6, characterized in that The device further comprises: The second acquisition module is used to obtain the target clarity; Furthermore, the first processing module includes: a calculation submodule, configured to calculate a horizontal sampling interpolation rate and a vertical sampling interpolation rate according to the target resolution C×D and the original resolution A×E; and The interpolation submodule is used to interpolate each of the single-channel images according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition to obtain a resolution of Four single-channel images of .

8. The device according to claim 7, wherein The computing submodule is specifically configured to: performing interpolation processing on the edge region of each single-channel image according to the horizontal sampling interpolation rate, the vertical sampling interpolation rate, and the target definition, wherein the edge region includes the first row, the first column, the last row, and the last column; and According to the horizontal sampling interpolation rate, the vertical sampling interpolation rate and the target definition, a bicubic interpolation method is used to perform interpolation processing on other areas of the single-channel image.

9. The device according to claim 8, wherein The computing submodule is specifically configured to: determining a plurality of first positions to be interpolated in the first row and the last row according to the horizontal sampling interpolation rate, and calculating a first pixel value of each first position to be interpolated according to the target definition and pixel values ​​of two pixels adjacent to each first position to be interpolated; and A plurality of second positions to be interpolated in the first column and the last column are determined according to the vertical sampling interpolation rate, and a second pixel value of each second position to be interpolated is calculated according to the target definition and the pixel values ​​of two pixels adjacent to each second position to be interpolated.

10. The device according to any one of claims 6 to 9, characterized in that The combined module is specifically used for: Set the resolution to The four single-channel images are combined according to the Bayer array to obtain a RAW format image with a resolution of C×D.

11. A computer device, characterized in that: including processor and memory; The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method according to any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images