An image processing method and apparatus

Abstract

This application provides an image processing method and apparatus, applied in the field of image processing. The method includes: acquiring multiple single-channel images contained in an input image; scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image, to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; stitching the multiple scaled images to obtain a single-channel original image; calculating the pixel value of each pixel in the original image in other channels based on the channel to which the pixel value belongs and the pixel values ​​of related pixels, to obtain a target image with the same number of channels as the input image. Because sufficient nonlinearity can be introduced during the stitching and restoration to multi-channel processing, the clarity of the generated target image can be improved; simultaneously, since no extensive training or other computational processes are required, the computational load of the entire process can be reduced.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method and apparatus. Background Technology

[0002] Super-resolution algorithms improve the resolution of an original image using hardware or software methods, that is, obtaining a high-resolution image from one or more low-resolution images. Existing super-resolution algorithms are mainly divided into two categories: traditional methods and deep learning methods. Traditional methods mainly include interpolation methods, sparse representation methods, local embedding methods, and example-based methods, while deep learning methods mainly include various deep learning methods based on neural network models. However, super-resolution images obtained using traditional methods have lower clarity, while super-resolution images obtained using deep learning methods involve significantly higher computational costs. Summary of the Invention

[0003] The purpose of this application is to provide an image processing method and apparatus to solve the technical problem of how to reduce the computational load of the resolution improvement process while ensuring the clarity of the generated image.

[0004] In a first aspect, embodiments of this application provide an image processing method, comprising: acquiring multiple single-channel images contained in an input image; scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image, to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; stitching the multiple scaled images together to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; calculating the pixel value of each pixel in other channels according to the channel to which the pixel value belongs and the pixel values ​​of related pixels in the original image, to obtain a target image with the same number of channels as the input image. In the above scheme, the input image at the initial resolution is separated into multiple single-channel images, and the scaled single-channel images are stitched together. Then, the stitched original image is restored to a multi-channel image, and the resulting target image is the image that meets the target resolution. Since sufficient nonlinearity can be introduced during the stitching and restoration to multi-channel processes, the clarity of the generated target image can be improved; at the same time, since no large amount of training or other computational processes are required, the computational load of the entire process can be reduced.

[0005] In an optional implementation, the channel includes a first type of channel, a second type of channel, and a third type of channel, and the multiple single-channel images include two first images corresponding to the first type of channel, one second image corresponding to the second type of channel, and one third image corresponding to the third type of channel.

[0006] In an optional implementation, stitching the multiple scaled images to obtain a single-channel original image includes: stitching pixels at the same location in the scaled first image, the scaled second image, and the scaled third image into a pixel block at the corresponding location in the original image; wherein each pixel block includes four pixels, with the two pixels on one diagonal of each pixel block being the pixels of the scaled second image and the scaled third image at that location, and the two pixels on the other diagonal being the pixels of the scaled first image at that location. In the above scheme, by stitching the scaled single-channel images into the original image, sufficient nonlinearity can be introduced, thus improving the clarity of the generated target image.

[0007] In an optional implementation, pixels in the original image whose pixel values ​​belong to the first channel are designated as first pixels, pixels in the original image whose pixel values ​​belong to the second channel are designated as second pixels, and pixels in the original image whose pixel values ​​belong to the third channel are designated as third pixels. The step of calculating the pixel values ​​of each pixel in the original image in other channels based on the channel to which its pixel value belongs and the pixel values ​​of pixels associated with that pixel, to obtain a target image with the same number of channels as the input image, includes: traversing the second and third pixels in the original image and determining the interpolation direction within a processing window centered on each second or third pixel; calculating the pixel value of the second or third pixel in the first channel based on the interpolation direction and the pixel values ​​of pixels associated with the second or third pixel within the processing window centered on that pixel, to obtain a first intermediate image. The process involves iterating through the first intermediate image to find the fourth pixel corresponding to the second pixel position and the fifth pixel corresponding to the third pixel position, and determining the interpolation direction within the processing window centered on each fourth or fifth pixel. Based on the interpolation direction and the pixel values ​​of the pixels related to the fourth or fifth pixel within the processing window centered on the first pixel, the process calculates the pixel value of the fourth pixel in the third channel or the pixel value of the fifth pixel in the second channel to obtain the second intermediate image. Then, the process involves iterating through the second intermediate image to find the sixth pixel corresponding to the first pixel position, and determining the interpolation direction within the processing window centered on each sixth pixel. Based on the interpolation direction and the pixel values ​​of the pixels related to the sixth pixel within the processing window centered on the sixth pixel, the process calculates the pixel values ​​of the sixth pixel in the second and third channels to obtain the target image. In the above scheme, by sequentially traversing the pixels in the original image that belong to single-channel pixel values ​​and inserting pixel values ​​from other channels, a multi-channel image can be restored from the original image. Since sufficient nonlinearity can be introduced during the insertion process, the clarity of the generated target image can be improved. At the same time, since no large amount of training and other computational processes are required, the computational load of the entire process can be reduced.

[0008] In an optional implementation, the step of traversing the second and third pixels in the original image and determining the interpolation direction within a processing window centered on each second or third pixel includes: calculating the horizontal gradient and the vertical gradient within the processing window; and determining the direction corresponding to the smaller value of the horizontal gradient and the vertical gradient as the interpolation direction.

[0009] In an optional implementation, the step of traversing the fourth pixel point corresponding to the position of the second pixel point and the fifth pixel point corresponding to the position of the third pixel point in the first intermediate image, and determining the interpolation direction within the processing window range centered on each fourth pixel point or fifth pixel point, includes: calculating the gradient of two diagonal directions within the processing window range; and determining the direction corresponding to the smaller value of the two diagonal direction gradients as the interpolation direction.

[0010] In an optional implementation, scaling the multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple scaled images includes: calculating a scaling factor based on the target resolution, the initial resolution, and the total number of single-channel images; and scaling the single-channel images according to the scaling factor to obtain the scaled images. In the above scheme, during the scaling process of the single-channel images, a corresponding scaling factor can be calculated based on the target resolution and the initial resolution to appropriately scale the single-channel images according to the scaling factor.

[0011] In an optional implementation, when the scaling factor is greater than 1, scaling the single-channel image according to the scaling factor to obtain the scaled image includes: for at least one image unit in the single-channel image, if the total number M of pixels to be inserted supports uniform interpolation, then M pixels to be inserted are uniformly inserted between every two original pixels in the image unit; if the total number M of pixels to be inserted does not support uniform interpolation, then N pixels to be inserted are uniformly inserted between every two original pixels in the image unit; wherein, N is a positive integer less than M, and M-N+1 is less than the number of original pixels in the image unit; calculating the pixel value to be inserted between every two adjacent pixels in the image unit after inserting the N pixels to be inserted; taking the distance between the line formed by the pixel value to be inserted and the pixel value of the original pixel, taking the pixel value to be inserted corresponding to the smallest MN distances among the calculated distances as the pixels of the MN pixels to be inserted, and inserting the MN pixels to be inserted between the pixels corresponding to the smallest MN distances. In the above scheme, when it is necessary to enlarge a single-channel image, interpolation can be used to scale the single-channel image. Specifically, when calculating the pixel value corresponding to the pixel to be inserted, the pixel values ​​of two adjacent pixels and the target sharpness can be considered to reduce the loss of sharpness during interpolation. Furthermore, for pixels to be inserted that do not have uniform interpolation values, they can be distributed as close as possible to or around the polyline to increase the overall variance.

[0012] In an optional implementation, when the scaling factor is greater than 1, scaling the single-channel image according to the scaling factor to obtain the scaled image includes: inserting pixels between pixels in the single-channel image; wherein the pixel value of the inserted pixel is calculated based on the pixel values ​​of the two pixels adjacent to the inserted pixel and the target sharpness, and the pixel value closer to the center point of the image unit in the single-channel image has a larger proportion in the calculation process.

[0013] In an optional implementation, the image unit includes at least one of the following objects: the first row, first column, last row, and last column of the single-channel image.

[0014] Secondly, embodiments of this application provide an image processing apparatus, comprising: an acquisition module for acquiring multiple single-channel images contained in an input image; a scaling module for scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image, to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; a stitching module for stitching the multiple scaled images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and a generation module for calculating the pixel value of each pixel in other channels based on the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of pixels related to that pixel, to obtain a target image with the same number of channels as the input image. In the above scheme, the input image at the initial resolution is separated into multiple single-channel images, and the scaled single-channel images are stitched together. Then, the stitched original image is restored to a multi-channel image, and the resulting target image is the image that meets the target resolution. In particular, since sufficient nonlinearity can be introduced during the stitching and restoration to multi-channel, the clarity of the generated target image can be improved; at the same time, since no large amount of training and other computational processes are required, the computational load of the entire process can be reduced.

[0015] In an optional implementation, the channel includes a first type of channel, a second type of channel, and a third type of channel, and the multiple single-channel images include two first images corresponding to the first type of channel, one second image corresponding to the second type of channel, and one third image corresponding to the third type of channel.

[0016] In an optional implementation, the stitching module is specifically used to: stitch pixels at the same position in the scaled first image, the scaled second image, and the scaled third image into a pixel block at the corresponding position in the original image; wherein each pixel block includes four pixels, with the two pixels on one diagonal of each pixel block being the pixels of the scaled first image and the scaled third image at that position, and the two pixels on the other diagonal being the pixels of the scaled second image at that position. In the above scheme, the scaled single-channel image is stitched into the original image through the stitching process, thereby introducing sufficient nonlinearity and thus improving the clarity of the generated target image.

[0017] In an optional implementation, pixels in the original image whose pixel values ​​belong to the first channel are designated as first pixels, pixels in the original image whose pixel values ​​belong to the second channel are designated as second pixels, and pixels in the original image whose pixel values ​​belong to the third channel are designated as third pixels. The generation module is specifically configured to: traverse the second and third pixels in the original image and determine the interpolation direction within a processing window centered on each second or third pixel; calculate the pixel value of the second or third pixel in the first channel based on the interpolation direction and the pixel values ​​of pixels related to the second or third pixel within the processing window centered on the second or third pixel, thereby obtaining a first intermediate image; and traverse the fourth pixel in the first intermediate image corresponding to the position of the second pixel and the pixel value of the third pixel. The fifth pixel corresponding to the first pixel position is identified, and the interpolation direction within the processing window centered on each fourth or fifth pixel is determined. Based on the interpolation direction and the pixel values ​​of pixels related to the fourth or fifth pixel within the processing window, the pixel value of the fourth pixel in the third channel or the pixel value of the fifth pixel in the second channel is calculated to obtain a second intermediate image. The sixth pixel corresponding to the first pixel position in the second intermediate image is traversed, and the interpolation direction within the processing window centered on each sixth pixel is determined. Based on the interpolation direction and the pixel values ​​of pixels related to the sixth pixel within the processing window, the pixel values ​​of the sixth pixel in the second and third channels are calculated to obtain the target image. In the above scheme, by sequentially traversing pixels in the original image, including those belonging to single-channel pixel values, and inserting pixel values ​​from other channels, a multi-channel image can be restored from the original image. Since sufficient nonlinearity can be introduced during the insertion process, the clarity of the generated target image can be improved. Simultaneously, since a large amount of training and other computational processes are not required, the computational load of the entire process can be reduced.

[0018] In an optional implementation, the generation module is further configured to: calculate the horizontal gradient and the vertical gradient within the processing window; and determine the direction corresponding to the smaller value of the horizontal gradient and the vertical gradient as the interpolation direction.

[0019] In an optional implementation, the generation module is further configured to: calculate the gradients of two diagonal directions within the processing window; and determine the direction corresponding to the smaller value of the two diagonal gradients as the interpolation direction.

[0020] In an optional implementation, the scaling module is specifically used to: calculate a scaling factor based on the target resolution, the initial resolution, and the total number of single-channel images; and scale the single-channel images according to the scaling factor to obtain the scaled image. In the above scheme, during the scaling process of the single-channel image, a corresponding scaling factor can be calculated based on the target resolution and the initial resolution to appropriately scale the single-channel image according to the scaling factor.

[0021] In an optional implementation, when the scaling rate is greater than 1, the scaling module is further configured to: for at least one image unit in the single-channel image, if the total number M of pixels to be inserted supports uniform interpolation, then uniformly insert M pixels to be inserted between every two original pixels in the image unit; if the total number M of pixels to be inserted does not support uniform interpolation, then uniformly insert N pixels to be inserted between every two original pixels in the image unit; wherein N is a positive integer less than M, and M-N+1 is less than the number of original pixels in the image unit; calculate the pixel value to be inserted between every two adjacent pixels in the image unit after inserting the N pixels to be inserted; determine the distance between the line formed by the pixel value to be inserted and the pixel value of the original pixel, and take the pixel value to be inserted corresponding to the smallest MN distances among the calculated distances as the pixels of the MN pixels to be inserted, and insert the MN pixels to be inserted between the pixels corresponding to the smallest MN distances. In the above scheme, when it is necessary to enlarge the single-channel image, interpolation can be used to scale the single-channel image. In calculating the pixel value corresponding to the pixel to be inserted, the pixel values ​​of two adjacent pixels and the target sharpness can be used to reduce the loss of sharpness during the interpolation process. In addition, for the pixel to be inserted that cannot be uniformly interpolated, it can be distributed as much as possible on or around the polyline to increase the overall variance.

[0022] In an optional implementation, the scaling module is further configured to: insert pixels between pixels in the single-channel image; wherein the pixel value of the inserted pixel is calculated based on the pixel values ​​of the two pixels adjacent to the inserted pixel and the target sharpness, and the pixel value closer to the center point of the image unit in the single-channel image has a greater proportion in the calculation process.

[0023] In an optional implementation, the image unit includes at least one of the following objects: the first row, first column, last row, and last column of the single-channel image.

[0024] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other via the bus; the memory stores program instructions executable by the processor, and the processor can execute the image processing method as described in the first aspect by calling the program instructions.

[0025] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium that stores computer instructions, which, when executed by a computer, cause the computer to perform the image processing method as described in the first aspect.

[0026] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described in the first aspect. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating an image processing method provided in an embodiment of this application;

[0029] Figure 2 A schematic diagram of a pixel block provided in an embodiment of this application;

[0030] Figure 3 A schematic diagram of another pixel block provided in an embodiment of this application;

[0031] Figure 4 A flowchart illustrating the specific implementation of step S102 provided in this application embodiment;

[0032] Figure 5 This is a schematic diagram of pixels within a processing window area according to an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of another processing window range of pixels according to an embodiment of this application;

[0034] Figure 7 A structural block diagram of an image processing apparatus provided in an embodiment of this application;

[0035] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0037] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This image processing method can be executed by the electronic device provided in this embodiment. The image processing method may include the following:

[0038] Step S101: Obtain multiple single-channel images contained in the input image.

[0039] Step S102: Scale multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple scaled images.

[0040] Step S103: Stitch together multiple scaled images to obtain a single-channel original image. The resolution of the original image is the target resolution.

[0041] Step S104: Based on the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of the pixels related to that pixel, calculate the pixel value of that pixel in other channels to obtain a target image with the same number of channels as the input image.

[0042] Specifically, images can be categorized into single-channel images, three-channel images, four-channel images, etc. In a single-channel image, each pixel corresponds to only one pixel value, and that pixel value belongs to only one channel. For example, in a single-channel red image, each pixel's pixel value belongs to the red channel. In a three-channel image, each pixel corresponds to three pixel values, and these three pixel values ​​belong to the first, second, and third channels respectively. For example, in an RGB image, each pixel's three pixel values ​​belong to the red, green, and blue channels respectively.

[0043] In this embodiment, the electronic device can acquire multiple single-channel images contained in an input image in step S101. There are several ways for the electronic device to acquire multiple single-channel images. Taking an RGB image as an example, the electronic device can first acquire an input image and then separate it into three single-channel images, each corresponding to one of the red, green, and blue channels. Alternatively, the electronic device can directly receive three single-channel images corresponding to an input image sent by an external device or read from a cloud server; similarly, each single-channel image corresponds to one of the red, green, and blue channels.

[0044] Understandably, as one implementation method, the electronic device may acquire more than three single-channel images. For example, it may acquire four single-channel images, including two images corresponding to the first type of channel, one image corresponding to the second type of channel, and one image corresponding to the third type of channel. The two images corresponding to the first type of channel can be obtained by copying one of the images corresponding to the first type of channel. Alternatively, it may acquire five single-channel images, including two images corresponding to the first type of channel, one image corresponding to the second type of channel, and two images corresponding to the third type of channel. Similarly, the two images corresponding to the first type of channel or the second type of channel can be obtained by copying one of the images corresponding to the first type of channel or the second type of channel.

[0045] Before executing step S102, the electronic device can also acquire the initial resolution corresponding to the input image and the target resolution corresponding to the target image. In other words, the purpose of the image processing method provided in this application embodiment is to convert the input image at the initial resolution into a target image at the target resolution. Similar to the above embodiments, there are various ways for the electronic device to acquire the initial resolution and target resolution. For example, the electronic device can read the initial resolution or target resolution stored locally; the electronic device can read the initial resolution or target resolution stored from a cloud server; or the electronic device can also read the initial resolution or target resolution input by the user, etc. This application embodiment does not specifically limit these methods; those skilled in the art can make appropriate choices based on the actual situation.

[0046] It is understood that the target resolution provided in the embodiments of this application may be greater than the initial resolution.

[0047] In one implementation, in addition to obtaining the initial resolution and the target resolution, the electronic device can also obtain other parameters required in the process of executing the image processing method provided in the embodiments of this application, such as target sharpness, target image size, etc. Those skilled in the art can also make appropriate adjustments according to the actual situation.

[0048] Next, in step S102, the electronic device can perform scaling processing on multiple single-channel images according to the obtained target resolution and initial resolution, thereby obtaining a scaled image corresponding to each single-channel image. There are various scaling processing methods, such as: increasing the resolution of a single-channel image through interpolation; decreasing the resolution of a single-channel image by deleting some pixels; and increasing the resolution of a single-channel image through local embedding, etc.

[0049] It should be noted that subsequent embodiments will provide a detailed description of the specific implementation process for increasing the resolution of a single-channel image through interpolation and decreasing the resolution of a single-channel image by deleting some pixels, which will not be described here.

[0050] Then, after obtaining multiple scaled images corresponding to multiple single-channel images, the multiple scaled images can be stitched together in step S103 to obtain the original single-channel image, and the resolution of the original image is the same as the target resolution. There are various ways to stitch the scaled images depending on the number of single-channel images and the specific channel type, and this embodiment does not impose specific limitations on these methods.

[0051] Taking step S101 as an example, where four single-channel images are obtained, including two images corresponding to the first type of channel (first image), one image corresponding to the second type of channel (second image), and one image corresponding to the third type of channel (third image), step S103 may specifically include the following:

[0052] Pixels at the same location in the scaled first, second, and third images are stitched together to form a pixel block at the corresponding location in the original image. Each pixel block consists of four pixels: two pixels on one diagonal of each block, representing the corresponding pixels in the scaled second and third images at that location; and two pixels on the other diagonal, representing the corresponding pixels in the scaled first image at that location. Therefore, this stitching method allows pixel values ​​from different channels to be uniformly mixed, facilitating interpolation.

[0053] It is understandable that in the original image obtained after performing the above steps, each pixel block can be like... Figure 2 Or such as Figure 3 As shown. Among them, Figure 2 This is a schematic diagram of a pixel block provided in an embodiment of this application. Figure 3 This is a schematic diagram of another pixel block provided in an embodiment of the present application, including diagonally lined squares representing pixels in a scaled first image, vertically lined squares representing pixels in a scaled second image, and horizontally lined squares representing pixels in a scaled third image.

[0054] It should be noted that when the first type of channel is a green channel, the second type of channel is a red channel, and the third type of channel is a blue channel, such as... Figure 2 The pixel blocks shown are obtained by stitching together scaled images according to the Bayer pattern; alternatively, when the first type of channel is the red channel, the second type of channel is the green channel, and the third type of channel is the blue channel, they can also be stitched together as follows: Figure 2 The splicing is done in the manner shown.

[0055] In the case of four single-channel images, to ensure that the resolution of the original image is the target resolution, the resolution of the scaled image obtained in step S102 above should be one-quarter of the target resolution. This is because, in the horizontal direction, since the number of pixels in the original image is twice the number of pixels in the scaled image, the scaled image resolution in the horizontal direction is half the resolution of the original image; similarly, in the vertical direction, since the number of pixels in the original image is twice the number of pixels in the scaled image, the scaled image resolution in the horizontal direction is also half the resolution of the original image.

[0056] Finally, after obtaining the original image with a single channel, the electronic device can complete the channel count of each pixel in the original image in step S104 to obtain a target image with the same number of channels as the input image. For example, if the input image is a three-channel image, then the pixel values ​​corresponding to the other two channels need to be added to each pixel in the original image so that each pixel satisfies the three-channel requirement.

[0057] It should be noted that subsequent embodiments will provide a detailed description of the specific implementation process for completing the channels of each pixel in the original image, which will not be described here.

[0058] Therefore, in this embodiment, the input image at the initial resolution is separated into multiple single-channel images, and the scaled single-channel images are stitched together. Then, the stitched original image is restored to a multi-channel image, resulting in the target image that meets the target resolution. Since sufficient nonlinearity can be introduced during the stitching and restoration to multi-channel processing, the clarity of the generated target image can be improved. Simultaneously, since no extensive training or other computational processes are required, the overall computational load can be reduced.

[0059] The specific implementation process of step S102 above is described below. Please refer to... Figure 4 , Figure 4 A flowchart illustrating a specific implementation of step S102 provided in this application embodiment. Step S102 may include the following:

[0060] Step S401: Calculate the scaling factor based on the target resolution, the initial resolution, and the total number of images in a single channel.

[0061] Step S402: Scale the single-channel image according to the scaling factor to obtain a scaled image.

[0062] Specifically, the scaling factor can be calculated based on the target resolution, the initial resolution, and the total number of images in a single channel. The scaling factor can include a single value, such as magnification by two times; or it can include two values, such as a horizontal scaling factor and a vertical scaling factor.

[0063] For example, assuming the target resolution is C×D, the initial resolution is A×E, and the total number of single-channel images is 4, the horizontal scaling factor can be calculated using the following formula:

[0064]

[0065] The vertical scaling factor can be calculated using the following formula:

[0066]

[0067] After calculating the scaling factor, if the scaling factor is greater than 1, it indicates that the single-channel image needs to be enlarged; if the scaling factor is less than 1, it indicates that the single-channel image needs to be reduced. The following describes in detail the specific implementation of step S402, taking scaling factors including horizontal and vertical scaling factors as an example.

[0068] As one implementation method, when the scaling factor is greater than 1, the resolution of a single-channel image can be increased by interpolation. In this case, step S402 may specifically include the following:

[0069] For at least one image unit in a single-channel image, if the total number M of pixels to be inserted supports uniform interpolation, then the M pixels to be inserted are uniformly inserted between every two original pixels in the image unit.

[0070] If the total number M of pixels to be inserted does not support uniform interpolation, then N pixels to be inserted are evenly inserted between every two original pixels in the image unit.

[0071] Calculate the value of the pixel to be inserted between every two adjacent pixels in the image unit after inserting N pixels.

[0072] The distance between the pixel value to be inserted and the pixel values ​​of the original pixels is used to calculate the distance between the original pixels and the line segment. The pixel values ​​corresponding to the smallest MN distances among these calculated distances are then used as the pixels to be inserted into the MN nodes. The original pixels refer to the pixels in the single-channel image before interpolation.

[0073] Where N is a positive integer less than M, and M-N+1 is less than the number of original pixels in an image unit. An image unit can be a row or a column in a single-channel image. The total number M of pixels to be inserted can be determined based on the scaling factor and the number of pixels in an image unit in a single-channel image.

[0074] For example, when the original number of pixels in an image unit is 3, the horizontal scaling factor is... In this case, for a row in a single-channel image, the total number of pixels to be inserted is M = 2. Since the size of M is equal to the number of pixels in the image unit minus 1, the total number of pixels to be inserted, M, supports uniform interpolation. Two pixels to be inserted can be directly and evenly inserted between every two original pixels in the image unit, that is, exactly one pixel is inserted between every two original pixels.

[0075] For example, when the number of original pixels in an image unit is 3, the horizontal scaling factor is... In a single-channel image, for one row, the total number of pixels to be inserted, M = 3. Since M is not equal to the number of pixels in the image unit minus 1, uniform interpolation is not supported. Therefore, two pixels to be inserted can be evenly inserted between every two original pixels in the image unit. Then, the pixel values ​​to be inserted between every two adjacent pixels in the image unit (four in total) after inserting the two pixels are calculated. The remaining pixel is then inserted between the pixel with the smallest distance from the line formed by the original pixel values ​​among the four pixel values. For example, if the distance between the calculated pixel value and the line formed by the first and second pixels is the smallest, then the remaining pixel is inserted between the first and second pixels.

[0076] As can be seen from the two examples above, during the interpolation process, uniform interpolation should be ensured as much as possible. If uniform interpolation is not possible, non-uniform interpolation can be used for a small number of pixels.

[0077] It should be noted that in the above embodiments, the pixel value of the inserted pixel can be calculated based on the pixel values ​​of the two adjacent pixels and the target sharpness. Furthermore, the pixel value of pixels closer to the center of the image unit can be given a larger proportion in the calculation process. This makes the newly inserted pixel closer to the median value. This non-uniform transition method can effectively reduce the transition band, thereby preventing excessive sharpness loss as seen in traditional interpolation methods, resulting in a higher sharpness of the final image.

[0078] For example, when the original number of pixels in an image unit is 3, the horizontal scaling factor is... For a single-channel image, in a given row, the total number of pixels to be inserted is M = 2. The first pixel to be inserted is placed between the first and second original pixels, and its pixel value can be determined using the following formula:

[0079]

[0080] Where Pixel2 is the pixel value of the first pixel to be inserted, α is the target sharpness, Pixel1 is the pixel value of the first original pixel, and Pixel3 is the pixel value of the second original pixel.

[0081] It is understood that all image units in a single-channel image can be magnified using the interpolation method described in the above embodiments; alternatively, some image units can be magnified using the interpolation method described in the above embodiments, while other image units can be magnified using other interpolation methods (e.g., bilinear interpolation, bicubic interpolation, etc.). As one implementation, the first row, first column, last row, and last column of a single-channel image can be magnified using the interpolation method described in the above embodiments, while other rows and columns in the single-channel image, excluding the first row, first column, last row, and last column, can be magnified using other interpolation methods. This is because, for pixels in the first row, first column, last row, and last column, there are no completely adjacent pixels in all four directions, therefore other interpolation methods cannot be used.

[0082] Therefore, in this embodiment of the application, when it is necessary to enlarge a single-channel image, interpolation can be used to scale the single-channel image. Specifically, when calculating the pixel value corresponding to the pixel to be inserted, the pixel values ​​of two adjacent pixels and the target sharpness can be considered to reduce the loss of sharpness during interpolation. Furthermore, for pixels to be inserted that do not have uniform interpolation values, they can be distributed as close as possible to or around the polyline to increase the overall variance.

[0083] As another implementation method, when the scaling factor is less than 1, the resolution of a single-channel image can be reduced by deleting some pixels. The specific implementation process is described below.

[0084] The following describes the specific implementation process of completing the channels for each pixel of the original image. Taking step S101 where four single-channel images are obtained, including two images corresponding to the first type of channel, one image corresponding to the second type of channel, and one image corresponding to the third type of channel, as an example, step S104 can specifically include the following:

[0085] Iterate through the second and third pixels in the original image and determine the interpolation direction within the processing window centered on each second or third pixel.

[0086] Based on the interpolation direction and the pixel values ​​of the pixels related to the second or third pixel within the processing window, the pixel values ​​of the second or third pixel in the first channel are calculated to obtain the first intermediate image.

[0087] Traverse the fourth pixel corresponding to the second pixel position and the fifth pixel corresponding to the third pixel position in the first intermediate image, and determine the interpolation direction within the processing window centered on each fourth or fifth pixel.

[0088] Based on the interpolation direction and the pixel values ​​of the pixels related to the fourth or fifth pixel within the processing window, the pixel value of the fourth pixel in the third channel or the pixel value of the fifth pixel in the second channel is calculated to obtain the second intermediate image.

[0089] Traverse the sixth pixel in the second intermediate image corresponding to the position of the first pixel, and determine the interpolation direction within the processing window centered on each sixth pixel.

[0090] Based on the interpolation direction and the pixel values ​​of the pixels related to the sixth pixel within the processing window, the pixel values ​​of the sixth pixel in the second and third channels are calculated to obtain the target image.

[0091] Specifically, for ease of description, pixels in the original image whose pixel values ​​belong to the first channel are named the first pixel; pixels in the original image whose pixel values ​​belong to the second channel are named the second pixel; pixels in the original image whose pixel values ​​belong to the third channel are named the third pixel; pixels in the first intermediate image whose pixel values ​​belong to both the first and second channels are named the fourth pixel; pixels in the original image whose pixel values ​​belong to both the first and third channels are named the fifth pixel; and pixels in the second intermediate image whose pixel values ​​belong to the first channel are named the sixth pixel.

[0092] In the process of supplementing the channels of pixels in the original image, we can iterate through every pixel in the original image. This iteration can be divided into three steps:

[0093] The first process involves a first pass through the image. When the second pixel in the original image is reached, the pixel value corresponding to the first channel is inserted at that pixel. Similarly, when the third pixel in the original image is reached, the pixel value corresponding to the first channel is inserted at that pixel. After passing through all the pixels, the first intermediate image is obtained.

[0094] The second process involves a second pass through the image. When the traversal reaches the fifth pixel in the first intermediate image, the pixel value corresponding to the third channel is inserted at that fourth pixel. Similarly, when the traversal reaches the fifth pixel in the first intermediate image, the pixel value corresponding to the second channel is inserted at that fifth pixel. After traversing all the pixels, the second intermediate image is obtained.

[0095] The third process involves a third pass through the image. When the traversal reaches the sixth pixel in the second intermediate image, the pixel value corresponding to the second channel and the pixel value corresponding to the third channel are inserted at that sixth pixel. After traversing all the pixels, the target image is obtained.

[0096] In the above three processes, when traversing to a pixel, the interpolation direction within the processing window centered on that pixel can be determined first. Then, based on the interpolation direction and the pixel values ​​of the pixels related to that pixel within the processing window, the inserted pixel value is determined.

[0097] As one implementation method, the step of determining the interpolation direction in the first and second processes may specifically include the following:

[0098] Calculate the horizontal and vertical gradients within the processing window.

[0099] The direction corresponding to the smaller value of the horizontal gradient and the vertical gradient is determined as the interpolation direction.

[0100] Since there are pixels corresponding to the first type of channel in the horizontal and vertical directions of the second pixel within the processing window, the horizontal and vertical gradients of the second pixel can be calculated based on the pixel values ​​of the pixels corresponding to the first type of channel.

[0101] Similarly, since there are pixels corresponding to the first type of channel in the horizontal and vertical directions of the third pixel within the processing window, the horizontal and vertical gradients of the third pixel can be calculated based on the pixel values ​​of the pixels corresponding to the first type of channel.

[0102] Similarly, since there are pixels corresponding to the second and third channels in the horizontal and vertical directions of the sixth pixel within the processing window, the horizontal and vertical gradients of the sixth pixel can be calculated based on the pixel values ​​of the pixels corresponding to the second and third channels.

[0103] As another implementation, the step of determining the interpolation direction in the third process may specifically include the following:

[0104] Calculate the gradients in the two diagonal directions within the processing window.

[0105] The direction corresponding to the smaller value of the gradients in the two diagonal directions is determined as the interpolation direction.

[0106] Since there are no pixels corresponding to the third type of channel in the horizontal and vertical directions of the fourth pixel within the processing window, but there are pixels corresponding to the third type of channel in the diagonal direction, the horizontal and vertical gradients of the fourth pixel can be calculated based on the pixel values ​​of the pixels corresponding to the third type of channel.

[0107] Similarly, since there are no pixels corresponding to the second type of channel in the horizontal and vertical directions of the fifth pixel within the processing window, but there are pixels corresponding to the second type of channel in the diagonal direction, the horizontal and vertical gradients of the fifth pixel can be calculated based on the pixel values ​​of the pixels corresponding to the second type of channel.

[0108] It should be noted that after calculating the pixel value of the pixel in other channels based on the interpolation direction and the pixel values ​​of the pixels related to the pixel within the processing window, it can be determined whether the calculated pixel value is within a certain range of pixel values. If the calculated pixel value is not within the above range, the boundary of the range can be taken as the pixel value of the pixel in other channels, thereby avoiding a large difference between the calculated pixel value and the pixel value of the whole image, resulting in low image clarity.

[0109] Furthermore, this application does not specifically limit the size of the processing window range or the pixel value range. For example, the processing window range can be a 5×5 range, a 6×6 range, etc.; the pixel value range can be 0-255, the minimum value among the pixels adjacent to the pixel, to the maximum value among the pixels adjacent to the pixel, etc. Those skilled in the art can make appropriate adjustments according to the actual situation.

[0110] It is understandable that for pixels located at the edge of the original image, the number of pixels within the processing window centered on that pixel is incomplete. In this case, zero-padding can be used to complete the pixels (i.e., setting the pixel value corresponding to the non-existent pixel to zero) or symmetric mapping can be used to complete the pixels (i.e., mapping the pixel value of the existing pixel to the pixel value of the corresponding non-existent pixel based on the symmetry between the non-existent and existing pixels).

[0111] The following example illustrates the specific implementation process of step S104 above.

[0112] Assume the first type of channel is the green channel, the second type of channel is the red channel, and the third type of channel is the blue channel; the processing window can be a 5×5 size range.

[0113] Please refer to Figure 5 and Figure 6 , Figure 5 This is a schematic diagram illustrating a processing window range of pixels according to an embodiment of this application. Figure 6This is a schematic diagram of pixels within another processing window range according to an embodiment of this application. The squares with diagonal lines represent pixels in the scaled first image, the squares with vertical lines represent pixels in the scaled second image, and the squares with horizontal lines represent pixels in the scaled third image; the pixels are numbered from 00 (first pixel in the first row) to 44 (last pixel in the last row).

[0114] During the first traversal, when traversing to Figure 5 When dealing with pixel 22, since pixel 22 is the first pixel, the pixel value corresponding to the green channel should be inserted. The process for calculating the pixel value corresponding to the green channel is as follows:

[0115] First, calculate the horizontal gradient of pixel 22 using the following formula:

[0116] Grad H =abs(R 22 -R 20 )+abs(R 22 -R 24 )+abs(R 20 -G 21 )+abs(G 21 -R 22 )+abs(R 22 -G 23 )+abs(G 23 -R 24 );

[0117] Among them, Grad H R is the horizontal gradient of pixel 22. 22 R represents the pixel value corresponding to pixel 22 and the red channel. 20 R represents the pixel value corresponding to pixel 20 and the red channel. 24 G represents the pixel value corresponding to pixel 24 and the red channel. 21 G represents the pixel value corresponding to pixel 21 in the green channel. 23 This refers to the pixel value corresponding to pixel 23 in the green channel.

[0118] Then, the vertical gradient of pixel 22 is calculated according to the following formula:

[0119] Grad V =abs(R 22 -R 02 )+abs(R 22 -R 42 )+abs(R 02 -G 12 )+abs(G 12 -R22 )+abs(R 22 -G 32 )+abs(G 32 -R 42 );

[0120] Among them, Grad V R is the horizontal gradient of pixel 22. 02 R represents the pixel value corresponding to pixel 02 and the red channel. 42 G represents the pixel value corresponding to pixel 42 and the red channel. 12 G represents the pixel value corresponding to pixel 12 in the green channel. 32 This represents the pixel value corresponding to pixel 32 and the green channel.

[0121] Next, we compare the horizontal gradient with the vertical gradient.

[0122] If the horizontal gradient is less than the vertical gradient, then the pixel value of pixel 22 corresponding to the green channel is calculated according to the following formula:

[0123]

[0124] Among them, Interp G This is the pixel value corresponding to pixel 22 and the green channel.

[0125] At this point, if the calculated pixel value is within [min(G 21 G 23 ), max(G 21 G 23 If the calculated pixel value is within the range of min(G), then the inserted pixel value is determined; if the calculated pixel value is less than min(G)... 21 G 23 If the value of the inserted pixel is min(G), then the value of the inserted pixel is determined to be min(G). 21 G 23 If the calculated pixel value is greater than max(G); 21 G 23 If the value of the inserted pixel is max(G), then the value of the inserted pixel is determined to be max(G). 21 G 23 ).

[0126] If the horizontal gradient is greater than the vertical gradient, then the pixel value corresponding to pixel 22 and the green channel is calculated according to the following formula:

[0127]

[0128] At this point, if the calculated pixel value is within [min(G 12 G 32 ), max(G 12G 32 If the calculated pixel value is within the range of min(G), then the inserted pixel value is determined; if the calculated pixel value is less than min(G)... 12 G 32 If the value of the inserted pixel is min(G), then the value of the inserted pixel is determined to be min(G). 12 G 32 If the calculated pixel value is greater than max(G); 12 G 32 If the value of the inserted pixel is max(G), then the value of the inserted pixel is determined to be max(G). 12 G 32 ).

[0129] During the second traversal, when traversing to Figure 5 When dealing with pixel 22, since pixel 22 is the fourth pixel, the pixel value corresponding to the blue channel should be inserted. The process for calculating the pixel value corresponding to the blue channel is as follows:

[0130] First, calculate the 45° gradient of pixel 22 using the following formula:

[0131] Grad 45 =abs(R 22 -R 00 )+abs(R 22 -R 44 )+abs(R 00 -G 11 )+abs(G 11 -R 22 )+abs(R 22 -G 33 )+abs(G 33 -R 44 );

[0132] Among them, Grad 45 R is the gradient of pixel 22 in the 45° direction. 22 R represents the pixel value corresponding to pixel 22 and the red channel. 00 R represents the pixel value corresponding to pixel 00 and the red channel. 44 G represents the pixel value corresponding to pixel 44 and the red channel. 11 G represents the pixel value corresponding to pixel 11 in the green channel. 33 This is the pixel value corresponding to pixel 33 in the green channel.

[0133] Then, the 135° gradient of pixel 22 is calculated according to the following formula:

[0134] Grad 135 =abs(R 22 -R 04 )+abs(R22 -R 40 )+abs(R 04 -G 13 )+abs(G 13 -R 22 )+abs(R 22 -G 31 )+abs(G 31 -R 40 );

[0135] Among them, Grad 135 R is the gradient of pixel 22 at 135°. 04 R represents the pixel value corresponding to pixel 04 and the red channel. 40 G represents the pixel value corresponding to pixel 40 and the red channel. 14 G represents the pixel value corresponding to pixel 14 in the green channel. 31 This refers to the pixel value corresponding to pixel 31 in the green channel.

[0136] Next, we compare the gradients in the 45° direction with those in the 135° direction.

[0137] If the gradient in the 45° direction is less than the gradient in the 135° direction, then the pixel value corresponding to pixel 22 and the blue channel is calculated according to the following formula:

[0138]

[0139] Among them, Interp B This represents the pixel value corresponding to pixel 22 in the blue channel.

[0140] At this point, if the calculated pixel value is within [min(B 11 B 33 ), max(S 11 B 33 If the calculated pixel value is within the range of min(B), then the inserted pixel value is determined; if the calculated pixel value is less than min(B) 11 B 33 If the value of the inserted pixel is min(B), then the value of the inserted pixel is determined to be min(B). 11 B 33 If the calculated pixel value is greater than max(B) 11 B 33 If ), then the inserted pixel value is determined to be max(B). 11 B 33 ).

[0141] If the gradient in the 45° direction is greater than the gradient in the 135° direction, then the pixel value corresponding to pixel 22 and the blue channel is calculated according to the following formula:

[0142]

[0143] At this point, if the calculated pixel value is within [min(B 13 B 31 ), max(B 13 B 31 If the calculated pixel value is within the range of min(B), then the inserted pixel value is determined; if the calculated pixel value is less than min(B) 13 B 31 If the value of the inserted pixel is min(B), then the value of the inserted pixel is determined to be min(B). 13 B 31 If the calculated pixel value is greater than max(B) 13 B 31 If ), then the inserted pixel value is determined to be max(B). 13 B 31 ).

[0144] During the third traversal, when traversing to Figure 6 When dealing with pixel 22, since pixel 22 is the sixth pixel, the pixel value corresponding to the red channel and the pixel value corresponding to the blue channel should be inserted. The process for calculating the pixel value corresponding to the red channel is as follows:

[0145] First, calculate the horizontal gradient of pixel 22 using the following formula:

[0146] Grad H =abs(G 22 -G 20 )+abs(G 22 -G 24 )+abs(G 20 -G 21 )+abs(G 21 -G 22 )+abs(G 22 -G 23 )+abs(G 23 -G 24 );

[0147] Among them, Grad H G is the horizontal gradient of pixel 22. 22 G represents the pixel value corresponding to pixel 22 in the green channel. 20 G represents the pixel value corresponding to pixel 20 in the green channel. 24 G represents the pixel value corresponding to pixel 24 in the green channel. 21 G represents the pixel value corresponding to pixel 21 in the green channel. 23 This refers to the pixel value corresponding to pixel 23 in the green channel.

[0148] Then, the vertical gradient of pixel 22 is calculated according to the following formula:

[0149] Grad V =abs(G 22 -G 02 )+abs(G 22 -G 42 )+abs(G 02 -G 12 )+abs(G 12 -G 22 )+abs(G 22 -G 32 )+abs(G 32 -G 42 );

[0150] Among them, Grad V G is the vertical gradient of pixel 22. 02 G represents the pixel value corresponding to pixel 02 in the green channel. 42 G represents the pixel value corresponding to pixel 42 and the green channel. 12 G represents the pixel value corresponding to pixel 12 in the green channel. 32 This represents the pixel value corresponding to pixel 32 and the green channel.

[0151] Next, we compare the horizontal gradient with the vertical gradient.

[0152] If the horizontal gradient is less than the vertical gradient, then the pixel value of pixel 22 corresponding to the blue channel is calculated according to the following formula:

[0153]

[0154] Among them, Interp R This represents the pixel value corresponding to pixel 22 in the blue channel.

[0155] At this point, if the calculated pixel value is within [min(R 21 R 23 ), max(R 21 R 23 If the calculated pixel value is within the range of min(R), then the inserted pixel value is determined; if the calculated pixel value is less than min(R)... 21 R 23 If ), then the inserted pixel value is determined to be min(R). 21 R 23 If the calculated pixel value is greater than max(R) 21 R 23 If ), then the value of the inserted pixel is determined to be max(R). 21 R 23 ).

[0156] If the horizontal gradient is greater than the vertical gradient, then the pixel value of pixel 22 corresponding to the blue channel is calculated according to the following formula:

[0157]

[0158] At this point, if the calculated pixel value is within [min(R 12 R 32 ), max(R 12 R 32 If the calculated pixel value is within the range of min(R), then the inserted pixel value is determined; if the calculated pixel value is less than min(R)... 12 R 32 If ), then the inserted pixel value is determined to be min(R). 12 R 32 If the calculated pixel value is greater than max(R) 12 R 32 If ), then the value of the inserted pixel is determined to be max(R). 12 R 32 ).

[0159] After the above three traversals, a three-channel target image can be obtained.

[0160] It should be noted that the above example only introduces the interpolation method for some pixels. It is understood that those skilled in the art can use similar implementation methods to calculate the interpolation method for other pixels based on the above example, which will not be elaborated here.

[0161] Therefore, in this embodiment of the application, by sequentially traversing the pixels in the original image that include those belonging to single-channel pixel values ​​and inserting pixel values ​​from other channels, a multi-channel image can be restored from the original image. Since sufficient nonlinearity can be introduced during the insertion process, the clarity of the generated target image can be improved. At the same time, since no large amount of training and other computational processes are required, the computational load of the entire process can be reduced.

[0162] Furthermore, the image processing method provided in this application embodiment can be applied to super-resolution schemes, that is, the image resolution can be improved by utilizing the image processing method provided in this application embodiment.

[0163] Please refer to Figure 7 , Figure 7This application provides a structural block diagram of an image processing apparatus 700, which may include: an acquisition module 701 for acquiring multiple single-channel images contained in an input image; a scaling module 702 for scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image, to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; a stitching module 703 for stitching the multiple scaled images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and a generation module 704 for calculating the pixel value of each pixel in other channels based on the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of pixels related to that pixel, to obtain a target image with the same number of channels as the input image.

[0164] In this embodiment, the input image at the initial resolution is separated into multiple single-channel images, and the scaled single-channel images are stitched together. The stitched original image is then restored to a multi-channel image, resulting in the target image that meets the target resolution. By introducing sufficient nonlinearity during the stitching and restoration to multi-channel processing, the clarity of the generated target image can be improved. Furthermore, since extensive training and other computational processes are unnecessary, the overall computational load is reduced.

[0165] Furthermore, the channels include a first type of channel, a second type of channel, and a third type of channel, and the multiple single-channel images include two first images corresponding to the first type of channel, one second image corresponding to the second type of channel, and one third image corresponding to the third type of channel.

[0166] Furthermore, the stitching module 703 is specifically used to: stitch together pixels at the same position in the scaled first image, the scaled second image, and the scaled third image to form a pixel block at the corresponding position in the original image; wherein, each pixel block includes four pixels, the two pixels on one diagonal of each pixel block are the pixels of the scaled second image and the scaled third image at that position respectively, and the two pixels on the other diagonal are the pixels of the scaled first image at that position.

[0167] In this embodiment of the application, the scaled single-channel image is stitched together to form the original image through a stitching process, thereby introducing sufficient nonlinearity and thus improving the clarity of the generated target image.

[0168] Furthermore, in the original image, pixels whose pixel values ​​belong to the first channel are designated as first pixels, pixels whose pixel values ​​belong to the second channel are designated as second pixels, and pixels whose pixel values ​​belong to the third channel are designated as third pixels. The generation module 704 is specifically used to: traverse the second and third pixels in the original image and determine the interpolation direction within a processing window centered on each second or third pixel; calculate the pixel value of the second or third pixel in the first channel based on the interpolation direction and the pixel values ​​of pixels related to the second or third pixel within the processing window centered on the second or third pixel, thereby obtaining a first intermediate image; and traverse the fourth pixel in the first intermediate image corresponding to the position of the second pixel and the pixel value of the third pixel. The fifth pixel corresponding to the prime point position is determined, and the interpolation direction within the processing window range centered on each fourth or fifth pixel is determined. Based on the interpolation direction and the pixel values ​​of the pixels related to the fourth or fifth prime point within the processing window range, the pixel value of the fourth pixel in the third channel or the pixel value of the fifth pixel in the second channel is calculated to obtain the second intermediate image. The sixth pixel corresponding to the first pixel position in the second intermediate image is traversed, and the interpolation direction within the processing window range centered on each sixth pixel is determined. Based on the interpolation direction and the pixel values ​​of the pixels related to the sixth pixel within the processing window range, the pixel values ​​of the sixth pixel in the second and third channels are calculated to obtain the target image.

[0169] In this embodiment, by sequentially traversing the pixels in the original image that include those belonging to a single-channel pixel value and inserting pixel values ​​from other channels, a multi-channel image can be reconstructed from the original image. Since sufficient nonlinearity can be introduced during the insertion process, the clarity of the generated target image can be improved. At the same time, since no large amount of training and other computational processes are required, the computational load of the entire process can be reduced.

[0170] Furthermore, the generation module 704 is also used to: calculate the horizontal gradient and the vertical gradient within the processing window; and determine the direction corresponding to the smaller value of the horizontal gradient and the vertical gradient as the interpolation direction.

[0171] Furthermore, the generation module 704 is also used to: calculate the gradients of two diagonal directions within the processing window; and determine the direction corresponding to the smaller value of the two diagonal gradients as the interpolation direction.

[0172] Furthermore, the scaling module 702 is specifically used to: calculate a scaling factor based on the target resolution, the initial resolution, and the total number of single-channel images; and scale the single-channel images according to the scaling factor to obtain the scaled image.

[0173] In the embodiments of this application, during the scaling process of a single-channel image, the corresponding scaling factor can be calculated based on the target resolution and the initial resolution, so as to appropriately scale the single-channel image according to the scaling factor.

[0174] Furthermore, when the scaling rate is greater than 1, the scaling module 702 is further configured to: for at least one image unit in the single-channel image, if the total number M of pixels to be inserted supports uniform interpolation, then uniformly insert M pixels to be inserted between every two original pixels in the image unit; if the total number M of pixels to be inserted does not support uniform interpolation, then uniformly insert N pixels to be inserted between every two original pixels in the image unit; wherein, N is a positive integer less than M, and M-N+1 is less than the number of original pixels in the image unit; calculate the pixel value to be inserted between every two adjacent pixels in the image unit after inserting the N pixels to be inserted; determine the distance between the line formed by the pixel value to be inserted and the pixel value of the original pixel, and take the pixel value to be inserted corresponding to the smallest MN distances among the calculated distances as the pixels of the MN pixels to be inserted, and insert the MN pixels to be inserted between the pixels corresponding to the smallest MN distances.

[0175] In this embodiment, when it is necessary to enlarge a single-channel image, interpolation can be used to scale the single-channel image. Specifically, when calculating the pixel value corresponding to the pixel to be inserted, the pixel values ​​of two adjacent pixels and the target sharpness can be considered to reduce the loss of sharpness during interpolation. Furthermore, for pixels to be inserted that do not have uniform interpolation values, they can be distributed as close as possible to or around the polyline to increase the overall variance.

[0176] Furthermore, when the scaling rate is greater than 1, the scaling module 702 is also used to: insert pixels between pixels in the single-channel image; wherein the pixel value of the inserted pixel is calculated based on the pixel values ​​of the two pixels adjacent to the inserted pixel and the target sharpness, and the pixel value closer to the center point of the image unit in the single-channel image has a greater proportion in the calculation process.

[0177] Furthermore, the image unit includes at least one of the following objects: the first row, the first column, the last row, and the last column of the single-channel image.

[0178] Please refer to Figure 8 , Figure 8 This application provides a structural block diagram of an electronic device 800, which includes at least one processor 801, at least one communication interface 802, at least one memory 803, and at least one communication bus 804. The communication bus 804 enables direct communication between these components, the communication interface 802 facilitates signaling or data communication with other node devices, and the memory 803 stores machine-readable instructions executable by the processor 801. When the electronic device 800 is running, the processor 801 communicates with the memory 803 via the communication bus 804, and the machine-readable instructions, when invoked by the processor 801, execute the aforementioned image processing method.

[0179] For example, the processor 801 in this embodiment of the application can read a computer program from the memory 803 via the communication bus 804 and execute the computer program to implement the following method: Step S101: Acquire multiple single-channel images contained in the input image. Step S102: Scale the multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple scaled images. Step S103: Stitch the multiple scaled images to obtain a single-channel original image. The resolution of the original image is the target resolution. Step S104: Calculate the pixel values ​​of each pixel in the original image in other channels based on the channel to which the pixel value belongs and the pixel values ​​of pixels related to that pixel, to obtain a target image with the same number of channels as the input image.

[0180] The processor 801 can be an integrated circuit chip with signal processing capabilities. The processor 801 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0181] The memory 803 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0182] Understandable. Figure 8 The structure shown is for illustrative purposes only; the electronic device 800 may also include components that are more advanced than those shown. Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown. Figure 8 The components shown can be implemented using hardware, software, or a combination thereof. In the embodiments of this application, electronic device 800 can be, but is not limited to, physical devices such as desktop computers, laptops, smartphones, smart wearable devices, and in-vehicle devices, or virtual devices such as virtual machines. Furthermore, electronic device 800 is not necessarily a single device; it can be a combination of multiple devices, such as a server cluster, etc.

[0183] This application also provides a computer program product, including a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by the computer, the computer can perform the steps of the image processing method described in the above embodiments, such as: acquiring multiple single-channel images contained in an input image; scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; stitching the multiple scaled images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; calculating the pixel value of each pixel in other channels according to the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of pixels related to that pixel, to obtain a target image with the same number of channels as the input image.

[0184] This application also discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as: acquiring multiple single-channel images contained in an input image; scaling the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution; stitching the multiple scaled images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; calculating the pixel value of each pixel in other channels according to the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of pixels related to that pixel, to obtain a target image with the same number of channels as the input image.

[0185] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0186] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0188] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0190] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Obtain multiple single-channel images contained in the input image; The multiple single-channel images are scaled according to the target resolution and the initial resolution corresponding to the input image to obtain multiple scaled images; wherein the target resolution is greater than the initial resolution. The multiple scaled images are stitched together to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; Based on the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of the pixels related to that pixel, the pixel value of that pixel in other channels is calculated to obtain a target image with the same number of channels as the input image; The step of scaling the multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple scaled images includes: The scaling factor is calculated based on the target resolution, the initial resolution, and the total number of images in a single channel; The single-channel image is scaled according to the scaling rate to obtain the scaled image; When the scaling factor is greater than 1, scaling the single-channel image according to the scaling factor to obtain the scaled image includes: Pixels are inserted between pixels in the single-channel image; wherein the pixel value of the inserted pixel is calculated based on the pixel values ​​of the two pixels adjacent to the inserted pixel and the target sharpness, and the pixel value of the pixel closer to the center point of the image unit in the single-channel image has a larger proportion in the calculation process.

2. The image processing method according to claim 1, characterized in that, The channels include a first type of channel, a second type of channel, and a third type of channel. The multiple single-channel images include two first images corresponding to the first type of channel, one second image corresponding to the second type of channel, and one third image corresponding to the third type of channel.

3. The image processing method according to claim 2, characterized in that, The step of stitching together the multiple scaled images to obtain a single-channel original image includes: The pixels at the same position in the scaled first image, scaled second image, and scaled third image are stitched together to form a pixel block at the corresponding position in the original image; wherein, each pixel block includes four pixels, the two pixels on one diagonal of each pixel block are the pixels of the scaled second image and the scaled third image at that position respectively, and the two pixels on the other diagonal are the pixels of the scaled first image at that position respectively.

4. The image processing method according to claim 3, characterized in that, In the original image, pixels whose pixel values ​​belong to the first channel are designated as first pixels, pixels whose pixel values ​​belong to the second channel are designated as second pixels, and pixels whose pixel values ​​belong to the third channel are designated as third pixels. The step of calculating the pixel value of a pixel in other channels based on the channel to which the pixel value of each pixel in the original image belongs and the pixel values ​​of related pixels, to obtain a target image with the same number of channels as the input image, includes: Traverse the second and third pixels in the original image and determine the interpolation direction within the processing window centered on each second or third pixel. Based on the interpolation direction and the pixel values ​​of the pixels related to the second pixel or the third pixel within the processing window range, the pixel values ​​of the second pixel or the third pixel on the first channel are calculated to obtain the first intermediate image; Traverse the fourth pixel corresponding to the second pixel position and the fifth pixel corresponding to the third pixel position in the first intermediate image, and determine the interpolation direction within the processing window range centered on each fourth pixel or fifth pixel. Based on the interpolation direction and the pixel values ​​of the pixels related to the fourth pixel or the fifth pixel within the processing window range, calculate the pixel value of the fourth pixel in the third channel or the pixel value of the fifth pixel in the second channel to obtain the second intermediate image; Traverse the sixth pixel in the second intermediate image corresponding to the position of the first pixel, and determine the interpolation direction within the processing window range centered on each sixth pixel. Based on the interpolation direction and the pixel values ​​of the pixels related to the sixth pixel within the processing window, the pixel values ​​of the sixth pixel in the second and third channels are calculated to obtain the target image.

5. The image processing method according to claim 4, characterized in that, The step of traversing the second and third pixels in the original image and determining the interpolation direction within the processing window centered on each second or third pixel includes: Calculate the horizontal and vertical gradients within the processing window. The direction corresponding to the smaller value of the horizontal gradient and the vertical gradient is determined as the interpolation direction.

6. The image processing method according to claim 4, characterized in that, The step of traversing the fourth pixel corresponding to the second pixel position and the fifth pixel corresponding to the third pixel position in the first intermediate image, and determining the interpolation direction within the processing window range centered on each fourth or fifth pixel, includes: Calculate the gradients in the two diagonal directions within the processing window; The direction corresponding to the smaller value of the two diagonal gradient directions is determined as the interpolation direction.

7. The image processing method according to claim 1, characterized in that, When the scaling factor is greater than 1, scaling the single-channel image according to the scaling factor to obtain the scaled image includes: For at least one image unit in the single-channel image, if the total number M of pixels to be inserted supports uniform interpolation, then the M pixels to be inserted are uniformly inserted between every two original pixels in the image unit. If the total number M of pixels to be inserted does not support uniform interpolation, then N pixels to be inserted are uniformly inserted between every two original pixels in the image unit; where N is a positive integer less than M, and M-N+1 is less than the number of original pixels in the image unit. Calculate the pixel value to be inserted between every two adjacent pixels in the image unit after inserting the N pixels to be inserted; Calculate the distance between the pixel value to be inserted and the pixel value of the original pixel in the polyline. Select the pixel values ​​to be inserted corresponding to the smallest MN distances among the calculated distances as the pixels of the MN pixels to be inserted, and insert the MN pixels to be inserted between the pixels corresponding to the smallest MN distances.

8. The image processing method according to claim 7, characterized in that, The image unit includes at least one of the following objects: the first row, first column, last row, and last column of the single-channel image.

9. An electronic device, characterized in that, include: Processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the image processing method as described in any one of claims 1-8 by calling the program instructions.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to perform the image processing method as described in any one of claims 1-8.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the image processing method as described in any one of claims 1-8.

Images